github.com/artpar/rclone@v1.67.3/rclone.1 (about) 1 .\"t 2 .\" Automatically generated by Pandoc 2.9.2.1 3 .\" 4 .TH "rclone" "1" "Mar 10, 2024" "User Manual" "" 5 .hy 6 .SH Rclone syncs your files to cloud storage 7 .PP 8 .IP \[bu] 2 9 About rclone 10 .IP \[bu] 2 11 What can rclone do for you? 12 .IP \[bu] 2 13 What features does rclone have? 14 .IP \[bu] 2 15 What providers does rclone support? 16 .IP \[bu] 2 17 Download (https://rclone.org/downloads/) 18 .IP \[bu] 2 19 Install (https://rclone.org/install/) 20 .IP \[bu] 2 21 Donate. (https://rclone.org/donate/) 22 .SS About rclone 23 .PP 24 Rclone is a command-line program to manage files on cloud storage. 25 It is a feature-rich alternative to cloud vendors\[aq] web storage 26 interfaces. 27 Over 70 cloud storage products support rclone including S3 object 28 stores, business & consumer file storage services, as well as standard 29 transfer protocols. 30 .PP 31 Rclone has powerful cloud equivalents to the unix commands rsync, cp, 32 mv, mount, ls, ncdu, tree, rm, and cat. 33 Rclone\[aq]s familiar syntax includes shell pipeline support, and 34 \f[C]--dry-run\f[R] protection. 35 It is used at the command line, in scripts or via its API. 36 .PP 37 Users call rclone \f[I]\[dq]The Swiss army knife of cloud 38 storage\[dq]\f[R], and \f[I]\[dq]Technology indistinguishable from 39 magic\[dq]\f[R]. 40 .PP 41 Rclone really looks after your data. 42 It preserves timestamps and verifies checksums at all times. 43 Transfers over limited bandwidth; intermittent connections, or subject 44 to quota can be restarted, from the last good file transferred. 45 You can check (https://rclone.org/commands/rclone_check/) the integrity 46 of your files. 47 Where possible, rclone employs server-side transfers to minimise local 48 bandwidth use and transfers from one provider to another without using 49 local disk. 50 .PP 51 Virtual backends wrap local and cloud file systems to apply 52 encryption (https://rclone.org/crypt/), 53 compression (https://rclone.org/compress/), 54 chunking (https://rclone.org/chunker/), 55 hashing (https://rclone.org/hasher/) and 56 joining (https://rclone.org/union/). 57 .PP 58 Rclone mounts (https://rclone.org/commands/rclone_mount/) any local, 59 cloud or virtual filesystem as a disk on Windows, macOS, linux and 60 FreeBSD, and also serves these over 61 SFTP (https://rclone.org/commands/rclone_serve_sftp/), 62 HTTP (https://rclone.org/commands/rclone_serve_http/), 63 WebDAV (https://rclone.org/commands/rclone_serve_webdav/), 64 FTP (https://rclone.org/commands/rclone_serve_ftp/) and 65 DLNA (https://rclone.org/commands/rclone_serve_dlna/). 66 .PP 67 Rclone is mature, open-source software originally inspired by rsync and 68 written in Go (https://golang.org). 69 The friendly support community is familiar with varied use cases. 70 Official Ubuntu, Debian, Fedora, Brew and Chocolatey repos. 71 include rclone. 72 For the latest version downloading from 73 rclone.org (https://rclone.org/downloads/) is recommended. 74 .PP 75 Rclone is widely used on Linux, Windows and Mac. 76 Third-party developers create innovative backup, restore, GUI and 77 business process solutions using the rclone command line or API. 78 .PP 79 Rclone does the heavy lifting of communicating with cloud storage. 80 .SS What can rclone do for you? 81 .PP 82 Rclone helps you: 83 .IP \[bu] 2 84 Backup (and encrypt) files to cloud storage 85 .IP \[bu] 2 86 Restore (and decrypt) files from cloud storage 87 .IP \[bu] 2 88 Mirror cloud data to other cloud services or locally 89 .IP \[bu] 2 90 Migrate data to the cloud, or between cloud storage vendors 91 .IP \[bu] 2 92 Mount multiple, encrypted, cached or diverse cloud storage as a disk 93 .IP \[bu] 2 94 Analyse and account for data held on cloud storage using 95 lsf (https://rclone.org/commands/rclone_lsf/), 96 ljson (https://rclone.org/commands/rclone_lsjson/), 97 size (https://rclone.org/commands/rclone_size/), 98 ncdu (https://rclone.org/commands/rclone_ncdu/) 99 .IP \[bu] 2 100 Union (https://rclone.org/union/) file systems together to present 101 multiple local and/or cloud file systems as one 102 .SS Features 103 .IP \[bu] 2 104 Transfers 105 .RS 2 106 .IP \[bu] 2 107 MD5, SHA1 hashes are checked at all times for file integrity 108 .IP \[bu] 2 109 Timestamps are preserved on files 110 .IP \[bu] 2 111 Operations can be restarted at any time 112 .IP \[bu] 2 113 Can be to and from network, e.g. 114 two different cloud providers 115 .IP \[bu] 2 116 Can use multi-threaded downloads to local disk 117 .RE 118 .IP \[bu] 2 119 Copy (https://rclone.org/commands/rclone_copy/) new or changed files to 120 cloud storage 121 .IP \[bu] 2 122 Sync (https://rclone.org/commands/rclone_sync/) (one way) to make a 123 directory identical 124 .IP \[bu] 2 125 Bisync (https://rclone.org/bisync/) (two way) to keep two directories in 126 sync bidirectionally 127 .IP \[bu] 2 128 Move (https://rclone.org/commands/rclone_move/) files to cloud storage 129 deleting the local after verification 130 .IP \[bu] 2 131 Check (https://rclone.org/commands/rclone_check/) hashes and for 132 missing/extra files 133 .IP \[bu] 2 134 Mount (https://rclone.org/commands/rclone_mount/) your cloud storage as 135 a network disk 136 .IP \[bu] 2 137 Serve (https://rclone.org/commands/rclone_serve/) local or remote files 138 over 139 HTTP (https://rclone.org/commands/rclone_serve_http/)/WebDav (https://rclone.org/commands/rclone_serve_webdav/)/FTP (https://rclone.org/commands/rclone_serve_ftp/)/SFTP (https://rclone.org/commands/rclone_serve_sftp/)/DLNA (https://rclone.org/commands/rclone_serve_dlna/) 140 .IP \[bu] 2 141 Experimental Web based GUI (https://rclone.org/gui/) 142 .SS Supported providers 143 .PP 144 (There are many others, built on standard protocols such as WebDAV or 145 S3, that work out of the box.) 146 .IP \[bu] 2 147 1Fichier 148 .IP \[bu] 2 149 Akamai Netstorage 150 .IP \[bu] 2 151 Alibaba Cloud (Aliyun) Object Storage System (OSS) 152 .IP \[bu] 2 153 Amazon S3 154 .IP \[bu] 2 155 Backblaze B2 156 .IP \[bu] 2 157 Box 158 .IP \[bu] 2 159 Ceph 160 .IP \[bu] 2 161 China Mobile Ecloud Elastic Object Storage (EOS) 162 .IP \[bu] 2 163 Arvan Cloud Object Storage (AOS) 164 .IP \[bu] 2 165 Citrix ShareFile 166 .IP \[bu] 2 167 Cloudflare R2 168 .IP \[bu] 2 169 DigitalOcean Spaces 170 .IP \[bu] 2 171 Digi Storage 172 .IP \[bu] 2 173 Dreamhost 174 .IP \[bu] 2 175 Dropbox 176 .IP \[bu] 2 177 Enterprise File Fabric 178 .IP \[bu] 2 179 Fastmail Files 180 .IP \[bu] 2 181 FTP 182 .IP \[bu] 2 183 Google Cloud Storage 184 .IP \[bu] 2 185 Google Drive 186 .IP \[bu] 2 187 Google Photos 188 .IP \[bu] 2 189 HDFS 190 .IP \[bu] 2 191 Hetzner Storage Box 192 .IP \[bu] 2 193 HiDrive 194 .IP \[bu] 2 195 HTTP 196 .IP \[bu] 2 197 ImageKit 198 .IP \[bu] 2 199 Internet Archive 200 .IP \[bu] 2 201 Jottacloud 202 .IP \[bu] 2 203 IBM COS S3 204 .IP \[bu] 2 205 IDrive e2 206 .IP \[bu] 2 207 IONOS Cloud 208 .IP \[bu] 2 209 Koofr 210 .IP \[bu] 2 211 Leviia Object Storage 212 .IP \[bu] 2 213 Liara Object Storage 214 .IP \[bu] 2 215 Linkbox 216 .IP \[bu] 2 217 Linode Object Storage 218 .IP \[bu] 2 219 Mail.ru Cloud 220 .IP \[bu] 2 221 Memset Memstore 222 .IP \[bu] 2 223 Mega 224 .IP \[bu] 2 225 Memory 226 .IP \[bu] 2 227 Microsoft Azure Blob Storage 228 .IP \[bu] 2 229 Microsoft Azure Files Storage 230 .IP \[bu] 2 231 Microsoft OneDrive 232 .IP \[bu] 2 233 Minio 234 .IP \[bu] 2 235 Nextcloud 236 .IP \[bu] 2 237 OVH 238 .IP \[bu] 2 239 Blomp Cloud Storage 240 .IP \[bu] 2 241 OpenDrive 242 .IP \[bu] 2 243 OpenStack Swift 244 .IP \[bu] 2 245 Oracle Cloud Storage Swift 246 .IP \[bu] 2 247 Oracle Object Storage 248 .IP \[bu] 2 249 ownCloud 250 .IP \[bu] 2 251 pCloud 252 .IP \[bu] 2 253 Petabox 254 .IP \[bu] 2 255 PikPak 256 .IP \[bu] 2 257 premiumize.me 258 .IP \[bu] 2 259 put.io 260 .IP \[bu] 2 261 Proton Drive 262 .IP \[bu] 2 263 QingStor 264 .IP \[bu] 2 265 Qiniu Cloud Object Storage (Kodo) 266 .IP \[bu] 2 267 Quatrix by Maytech 268 .IP \[bu] 2 269 Rackspace Cloud Files 270 .IP \[bu] 2 271 rsync.net 272 .IP \[bu] 2 273 Scaleway 274 .IP \[bu] 2 275 Seafile 276 .IP \[bu] 2 277 Seagate Lyve Cloud 278 .IP \[bu] 2 279 SeaweedFS 280 .IP \[bu] 2 281 SFTP 282 .IP \[bu] 2 283 Sia 284 .IP \[bu] 2 285 SMB / CIFS 286 .IP \[bu] 2 287 StackPath 288 .IP \[bu] 2 289 Storj 290 .IP \[bu] 2 291 Synology 292 .IP \[bu] 2 293 SugarSync 294 .IP \[bu] 2 295 Tencent Cloud Object Storage (COS) 296 .IP \[bu] 2 297 Uptobox 298 .IP \[bu] 2 299 Wasabi 300 .IP \[bu] 2 301 WebDAV 302 .IP \[bu] 2 303 Yandex Disk 304 .IP \[bu] 2 305 Zoho WorkDrive 306 .IP \[bu] 2 307 The local filesystem 308 .SS Virtual providers 309 .PP 310 These backends adapt or modify other storage providers: 311 .IP \[bu] 2 312 Alias: Rename existing remotes 313 .IP \[bu] 2 314 Cache: Cache remotes (DEPRECATED) 315 .IP \[bu] 2 316 Chunker: Split large files 317 .IP \[bu] 2 318 Combine: Combine multiple remotes into a directory tree 319 .IP \[bu] 2 320 Compress: Compress files 321 .IP \[bu] 2 322 Crypt: Encrypt files 323 .IP \[bu] 2 324 Hasher: Hash files 325 .IP \[bu] 2 326 Union: Join multiple remotes to work together 327 .SS Links 328 .IP \[bu] 2 329 Home page (https://rclone.org/) 330 .IP \[bu] 2 331 GitHub project page for source and bug 332 tracker (https://github.com/artpar/artpar) 333 .IP \[bu] 2 334 Rclone Forum (https://forum.rclone.org) 335 .IP \[bu] 2 336 Downloads (https://rclone.org/downloads/) 337 .SH Install 338 .PP 339 Rclone is a Go program and comes as a single binary file. 340 .SS Quickstart 341 .IP \[bu] 2 342 Download (https://rclone.org/downloads/) the relevant binary. 343 .IP \[bu] 2 344 Extract the \f[C]rclone\f[R] executable, \f[C]rclone.exe\f[R] on 345 Windows, from the archive. 346 .IP \[bu] 2 347 Run \f[C]rclone config\f[R] to setup. 348 See rclone config docs (https://rclone.org/docs/) for more details. 349 .IP \[bu] 2 350 Optionally configure automatic execution. 351 .PP 352 See below for some expanded Linux / macOS / Windows instructions. 353 .PP 354 See the usage (https://rclone.org/docs/) docs for how to use rclone, or 355 run \f[C]rclone -h\f[R]. 356 .PP 357 Already installed rclone can be easily updated to the latest version 358 using the rclone 359 selfupdate (https://rclone.org/commands/rclone_selfupdate/) command. 360 .PP 361 See the release signing docs (https://rclone.org/release_signing/) for 362 how to verify signatures on the release. 363 .SS Script installation 364 .PP 365 To install rclone on Linux/macOS/BSD systems, run: 366 .IP 367 .nf 368 \f[C] 369 sudo -v ; curl https://rclone.org/install.sh | sudo bash 370 \f[R] 371 .fi 372 .PP 373 For beta installation, run: 374 .IP 375 .nf 376 \f[C] 377 sudo -v ; curl https://rclone.org/install.sh | sudo bash -s beta 378 \f[R] 379 .fi 380 .PP 381 Note that this script checks the version of rclone installed first and 382 won\[aq]t re-download if not needed. 383 .SS Linux installation 384 .SS Precompiled binary 385 .PP 386 Fetch and unpack 387 .IP 388 .nf 389 \f[C] 390 curl -O https://downloads.rclone.org/rclone-current-linux-amd64.zip 391 unzip rclone-current-linux-amd64.zip 392 cd rclone-*-linux-amd64 393 \f[R] 394 .fi 395 .PP 396 Copy binary file 397 .IP 398 .nf 399 \f[C] 400 sudo cp rclone /usr/bin/ 401 sudo chown root:root /usr/bin/rclone 402 sudo chmod 755 /usr/bin/rclone 403 \f[R] 404 .fi 405 .PP 406 Install manpage 407 .IP 408 .nf 409 \f[C] 410 sudo mkdir -p /usr/local/share/man/man1 411 sudo cp rclone.1 /usr/local/share/man/man1/ 412 sudo mandb 413 \f[R] 414 .fi 415 .PP 416 Run \f[C]rclone config\f[R] to setup. 417 See rclone config docs (https://rclone.org/docs/) for more details. 418 .IP 419 .nf 420 \f[C] 421 rclone config 422 \f[R] 423 .fi 424 .SS macOS installation 425 .SS Installation with brew 426 .IP 427 .nf 428 \f[C] 429 brew install rclone 430 \f[R] 431 .fi 432 .PP 433 NOTE: This version of rclone will not support \f[C]mount\f[R] any more 434 (see #5373 (https://github.com/artpar/artpar/issues/5373)). 435 If mounting is wanted on macOS, either install a precompiled binary or 436 enable the relevant option when installing from source. 437 .PP 438 Note that this is a third party installer not controlled by the rclone 439 developers so it may be out of date. 440 Its current version is as below. 441 .PP 442 [IMAGE: Homebrew 443 package (https://repology.org/badge/version-for-repo/homebrew/rclone.svg)] (https://repology.org/project/rclone/versions) 444 .SS Installation with MacPorts (#macos-macports) 445 .PP 446 On macOS, rclone can also be installed via 447 MacPorts (https://www.macports.org): 448 .IP 449 .nf 450 \f[C] 451 sudo port install rclone 452 \f[R] 453 .fi 454 .PP 455 Note that this is a third party installer not controlled by the rclone 456 developers so it may be out of date. 457 Its current version is as below. 458 .PP 459 [IMAGE: MacPorts 460 port (https://repology.org/badge/version-for-repo/macports/rclone.svg)] (https://repology.org/project/rclone/versions) 461 .PP 462 More information here (https://ports.macports.org/port/rclone/). 463 .SS Precompiled binary, using curl 464 .PP 465 To avoid problems with macOS gatekeeper enforcing the binary to be 466 signed and notarized it is enough to download with \f[C]curl\f[R]. 467 .PP 468 Download the latest version of rclone. 469 .IP 470 .nf 471 \f[C] 472 cd && curl -O https://downloads.rclone.org/rclone-current-osx-amd64.zip 473 \f[R] 474 .fi 475 .PP 476 Unzip the download and cd to the extracted folder. 477 .IP 478 .nf 479 \f[C] 480 unzip -a rclone-current-osx-amd64.zip && cd rclone-*-osx-amd64 481 \f[R] 482 .fi 483 .PP 484 Move rclone to your $PATH. 485 You will be prompted for your password. 486 .IP 487 .nf 488 \f[C] 489 sudo mkdir -p /usr/local/bin 490 sudo mv rclone /usr/local/bin/ 491 \f[R] 492 .fi 493 .PP 494 (the \f[C]mkdir\f[R] command is safe to run, even if the directory 495 already exists). 496 .PP 497 Remove the leftover files. 498 .IP 499 .nf 500 \f[C] 501 cd .. && rm -rf rclone-*-osx-amd64 rclone-current-osx-amd64.zip 502 \f[R] 503 .fi 504 .PP 505 Run \f[C]rclone config\f[R] to setup. 506 See rclone config docs (https://rclone.org/docs/) for more details. 507 .IP 508 .nf 509 \f[C] 510 rclone config 511 \f[R] 512 .fi 513 .SS Precompiled binary, using a web browser 514 .PP 515 When downloading a binary with a web browser, the browser will set the 516 macOS gatekeeper quarantine attribute. 517 Starting from Catalina, when attempting to run \f[C]rclone\f[R], a 518 pop-up will appear saying: 519 .IP 520 .nf 521 \f[C] 522 \[dq]rclone\[dq] cannot be opened because the developer cannot be verified. 523 macOS cannot verify that this app is free from malware. 524 \f[R] 525 .fi 526 .PP 527 The simplest fix is to run 528 .IP 529 .nf 530 \f[C] 531 xattr -d com.apple.quarantine rclone 532 \f[R] 533 .fi 534 .SS Windows installation 535 .SS Precompiled binary 536 .PP 537 Fetch the correct binary for your processor type by clicking on these 538 links. 539 If not sure, use the first link. 540 .IP \[bu] 2 541 Intel/AMD - 64 542 Bit (https://downloads.rclone.org/rclone-current-windows-amd64.zip) 543 .IP \[bu] 2 544 Intel/AMD - 32 545 Bit (https://downloads.rclone.org/rclone-current-windows-386.zip) 546 .IP \[bu] 2 547 ARM - 64 548 Bit (https://downloads.rclone.org/rclone-current-windows-arm64.zip) 549 .PP 550 Open this file in the Explorer and extract \f[C]rclone.exe\f[R]. 551 Rclone is a portable executable so you can place it wherever is 552 convenient. 553 .PP 554 Open a CMD window (or powershell) and run the binary. 555 Note that rclone does not launch a GUI by default, it runs in the CMD 556 Window. 557 .IP \[bu] 2 558 Run \f[C]rclone.exe config\f[R] to setup. 559 See rclone config docs (https://rclone.org/docs/) for more details. 560 .IP \[bu] 2 561 Optionally configure automatic execution. 562 .PP 563 If you are planning to use the rclone 564 mount (https://rclone.org/commands/rclone_mount/) feature then you will 565 need to install the third party utility WinFsp (https://winfsp.dev/) 566 also. 567 .SS Windows package manager (Winget) 568 .PP 569 Winget (https://learn.microsoft.com/en-us/windows/package-manager/) 570 comes pre-installed with the latest versions of Windows. 571 If not, update the App 572 Installer (https://www.microsoft.com/p/app-installer/9nblggh4nns1) 573 package from the Microsoft store. 574 .PP 575 To install rclone 576 .IP 577 .nf 578 \f[C] 579 winget install Rclone.Rclone 580 \f[R] 581 .fi 582 .PP 583 To uninstall rclone 584 .IP 585 .nf 586 \f[C] 587 winget uninstall Rclone.Rclone --force 588 \f[R] 589 .fi 590 .SS Chocolatey package manager 591 .PP 592 Make sure you have Choco (https://chocolatey.org/) installed 593 .IP 594 .nf 595 \f[C] 596 choco search rclone 597 choco install rclone 598 \f[R] 599 .fi 600 .PP 601 This will install rclone on your Windows machine. 602 If you are planning to use rclone 603 mount (https://rclone.org/commands/rclone_mount/) then 604 .IP 605 .nf 606 \f[C] 607 choco install winfsp 608 \f[R] 609 .fi 610 .PP 611 will install that too. 612 .PP 613 Note that this is a third party installer not controlled by the rclone 614 developers so it may be out of date. 615 Its current version is as below. 616 .PP 617 [IMAGE: Chocolatey 618 package (https://repology.org/badge/version-for-repo/chocolatey/rclone.svg)] (https://repology.org/project/rclone/versions) 619 .SS Scoop package manager 620 .PP 621 Make sure you have Scoop (https://scoop.sh/) installed 622 .IP 623 .nf 624 \f[C] 625 scoop install rclone 626 \f[R] 627 .fi 628 .PP 629 Note that this is a third party installer not controlled by the rclone 630 developers so it may be out of date. 631 Its current version is as below. 632 .PP 633 [IMAGE: Scoop 634 package (https://repology.org/badge/version-for-repo/scoop/rclone.svg)] (https://repology.org/project/rclone/versions) 635 .SS Package manager installation 636 .PP 637 Many Linux, Windows, macOS and other OS distributions package and 638 distribute rclone. 639 .PP 640 The distributed versions of rclone are often quite out of date and for 641 this reason we recommend one of the other installation methods if 642 possible. 643 .PP 644 You can get an idea of how up to date or not your OS distribution\[aq]s 645 package is here. 646 .PP 647 [IMAGE: Packaging 648 status (https://repology.org/badge/vertical-allrepos/rclone.svg?columns=3)] (https://repology.org/project/rclone/versions) 649 .SS Docker installation 650 .PP 651 The rclone developers maintain a docker image for 652 rclone (https://hub.docker.com/r/artpar/artpar). 653 .PP 654 These images are built as part of the release process based on a minimal 655 Alpine Linux. 656 .PP 657 The \f[C]:latest\f[R] tag will always point to the latest stable 658 release. 659 You can use the \f[C]:beta\f[R] tag to get the latest build from master. 660 You can also use version tags, e.g. 661 \f[C]:1.49.1\f[R], \f[C]:1.49\f[R] or \f[C]:1\f[R]. 662 .IP 663 .nf 664 \f[C] 665 $ docker pull artpar/artpar:latest 666 latest: Pulling from artpar/artpar 667 Digest: sha256:0e0ced72671989bb837fea8e88578b3fc48371aa45d209663683e24cfdaa0e11 668 \&... 669 $ docker run --rm artpar/artpar:latest version 670 rclone v1.49.1 671 - os/arch: linux/amd64 672 - go version: go1.12.9 673 \f[R] 674 .fi 675 .PP 676 There are a few command line options to consider when starting an rclone 677 Docker container from the rclone image. 678 .IP \[bu] 2 679 You need to mount the host rclone config dir at \f[C]/config/rclone\f[R] 680 into the Docker container. 681 Due to the fact that rclone updates tokens inside its config file, and 682 that the update process involves a file rename, you need to mount the 683 whole host rclone config dir, not just the single host rclone config 684 file. 685 .IP \[bu] 2 686 You need to mount a host data dir at \f[C]/data\f[R] into the Docker 687 container. 688 .IP \[bu] 2 689 By default, the rclone binary inside a Docker container runs with UID=0 690 (root). 691 As a result, all files created in a run will have UID=0. 692 If your config and data files reside on the host with a non-root 693 UID:GID, you need to pass these on the container start command line. 694 .IP \[bu] 2 695 If you want to access the RC interface (either via the API or the Web 696 UI), it is required to set the \f[C]--rc-addr\f[R] to \f[C]:5572\f[R] in 697 order to connect to it from outside the container. 698 An explanation about why this is necessary is present 699 here (https://web.archive.org/web/20200808071950/https://pythonspeed.com/articles/docker-connection-refused/). 700 .RS 2 701 .IP \[bu] 2 702 NOTE: Users running this container with the docker network set to 703 \f[C]host\f[R] should probably set it to listen to localhost only, with 704 \f[C]127.0.0.1:5572\f[R] as the value for \f[C]--rc-addr\f[R] 705 .RE 706 .IP \[bu] 2 707 It is possible to use \f[C]rclone mount\f[R] inside a userspace Docker 708 container, and expose the resulting fuse mount to the host. 709 The exact \f[C]docker run\f[R] options to do that might vary slightly 710 between hosts. 711 See, e.g. 712 the discussion in this 713 thread (https://github.com/moby/moby/issues/9448). 714 .RS 2 715 .PP 716 You also need to mount the host \f[C]/etc/passwd\f[R] and 717 \f[C]/etc/group\f[R] for fuse to work inside the container. 718 .RE 719 .PP 720 Here are some commands tested on an Ubuntu 18.04.3 host: 721 .IP 722 .nf 723 \f[C] 724 # config on host at \[ti]/.config/artpar/artpar.conf 725 # data on host at \[ti]/data 726 727 # add a remote interactively 728 docker run --rm -it \[rs] 729 --volume \[ti]/.config/rclone:/config/rclone \[rs] 730 --user $(id -u):$(id -g) \[rs] 731 artpar/artpar \[rs] 732 config 733 734 # make sure the config is ok by listing the remotes 735 docker run --rm \[rs] 736 --volume \[ti]/.config/rclone:/config/rclone \[rs] 737 --user $(id -u):$(id -g) \[rs] 738 artpar/artpar \[rs] 739 listremotes 740 741 # perform mount inside Docker container, expose result to host 742 mkdir -p \[ti]/data/mount 743 docker run --rm \[rs] 744 --volume \[ti]/.config/rclone:/config/rclone \[rs] 745 --volume \[ti]/data:/data:shared \[rs] 746 --user $(id -u):$(id -g) \[rs] 747 --volume /etc/passwd:/etc/passwd:ro --volume /etc/group:/etc/group:ro \[rs] 748 --device /dev/fuse --cap-add SYS_ADMIN --security-opt apparmor:unconfined \[rs] 749 artpar/artpar \[rs] 750 mount dropbox:Photos /data/mount & 751 ls \[ti]/data/mount 752 kill %1 753 \f[R] 754 .fi 755 .SS Snap installation 756 .PP 757 [IMAGE: Get it from the Snap 758 Store (https://snapcraft.io/static/images/badges/en/snap-store-black.svg)] (https://snapcraft.io/rclone) 759 .PP 760 Make sure you have Snapd 761 installed (https://snapcraft.io/docs/installing-snapd) 762 .IP 763 .nf 764 \f[C] 765 $ sudo snap install rclone 766 \f[R] 767 .fi 768 .PP 769 Due to the strict confinement of Snap, rclone snap cannot access real 770 /home/$USER/.config/rclone directory, default config path is as below. 771 .IP \[bu] 2 772 Default config directory: 773 .RS 2 774 .IP \[bu] 2 775 /home/$USER/snap/rclone/current/.config/rclone 776 .RE 777 .PP 778 Note: Due to the strict confinement of Snap, \f[C]rclone mount\f[R] 779 feature is \f[C]not\f[R] supported. 780 .PP 781 If mounting is wanted, either install a precompiled binary or enable the 782 relevant option when installing from source. 783 .PP 784 Note that this is controlled by community 785 maintainer (https://github.com/boukendesho/rclone-snap) not the rclone 786 developers so it may be out of date. 787 Its current version is as below. 788 .PP 789 [IMAGE: rclone (https://snapcraft.io/rclone/badge.svg)] (https://snapcraft.io/rclone) 790 .SS Source installation 791 .PP 792 Make sure you have git and Go (https://golang.org/) installed. 793 Go version 1.18 or newer is required, the latest release is recommended. 794 You can get it from your package manager, or download it from 795 golang.org/dl (https://golang.org/dl/). 796 Then you can run the following: 797 .IP 798 .nf 799 \f[C] 800 git clone https://github.com/artpar/artpar.git 801 cd rclone 802 go build 803 \f[R] 804 .fi 805 .PP 806 This will check out the rclone source in subfolder rclone, which you can 807 later modify and send pull requests with. 808 Then it will build the rclone executable in the same folder. 809 As an initial check you can now run \f[C]./rclone version\f[R] 810 (\f[C].\[rs]rclone version\f[R] on Windows). 811 .PP 812 Note that on macOS and Windows the 813 mount (https://rclone.org/commands/rclone_mount/) command will not be 814 available unless you specify an additional build tag \f[C]cmount\f[R]. 815 .IP 816 .nf 817 \f[C] 818 go build -tags cmount 819 \f[R] 820 .fi 821 .PP 822 This assumes you have a GCC compatible C compiler (GCC or Clang) in your 823 PATH, as it uses cgo (https://pkg.go.dev/cmd/cgo). 824 But on Windows, the cgofuse (https://github.com/winfsp/cgofuse) library 825 that the cmount implementation is based on, also supports building 826 without cgo (https://github.com/golang/go/wiki/WindowsDLLs), i.e. 827 by setting environment variable CGO_ENABLED to value 0 (static linking). 828 This is how the official Windows release of rclone is being built, 829 starting with version 1.59. 830 It is still possible to build with cgo on Windows as well, by using the 831 MinGW port of GCC, e.g. 832 by installing it in a MSYS2 (https://www.msys2.org) distribution (make 833 sure you install it in the classic mingw64 subsystem, the ucrt64 version 834 is not compatible). 835 .PP 836 Additionally, to build with mount on Windows, you must install the third 837 party utility WinFsp (https://winfsp.dev/), with the \[dq]Developer\[dq] 838 feature selected. 839 If building with cgo, you must also set environment variable CPATH 840 pointing to the fuse include directory within the WinFsp installation 841 (normally 842 \f[C]C:\[rs]Program Files (x86)\[rs]WinFsp\[rs]inc\[rs]fuse\f[R]). 843 .PP 844 You may add arguments \f[C]-ldflags -s\f[R] to omit symbol table and 845 debug information, making the executable file smaller, and 846 \f[C]-trimpath\f[R] to remove references to local file system paths. 847 The official rclone releases are built with both of these. 848 .IP 849 .nf 850 \f[C] 851 go build -trimpath -ldflags -s -tags cmount 852 \f[R] 853 .fi 854 .PP 855 If you want to customize the version string, as reported by the 856 \f[C]rclone version\f[R] command, you can set one of the variables 857 \f[C]fs.Version\f[R], \f[C]fs.VersionTag\f[R] (to keep default suffix 858 but customize the number), or \f[C]fs.VersionSuffix\f[R] (to keep 859 default number but customize the suffix). 860 This can be done from the build command, by adding to the 861 \f[C]-ldflags\f[R] argument value as shown below. 862 .IP 863 .nf 864 \f[C] 865 go build -trimpath -ldflags \[dq]-s -X github.com/artpar/artpar/fs.Version=v9.9.9-test\[dq] -tags cmount 866 \f[R] 867 .fi 868 .PP 869 On Windows, the official executables also have the version information, 870 as well as a file icon, embedded as binary resources. 871 To get that with your own build you need to run the following command 872 \f[B]before\f[R] the build command. 873 It generates a Windows resource system object file, with extension 874 \&.syso, e.g. 875 \f[C]resource_windows_amd64.syso\f[R], that will be automatically picked 876 up by future build commands. 877 .IP 878 .nf 879 \f[C] 880 go run bin/resource_windows.go 881 \f[R] 882 .fi 883 .PP 884 The above command will generate a resource file containing version 885 information based on the fs.Version variable in source at the time you 886 run the command, which means if the value of this variable changes you 887 need to re-run the command for it to be reflected in the version 888 information. 889 Also, if you override this version variable in the build command as 890 described above, you need to do that also when generating the resource 891 file, or else it will still use the value from the source. 892 .IP 893 .nf 894 \f[C] 895 go run bin/resource_windows.go -version v9.9.9-test 896 \f[R] 897 .fi 898 .PP 899 Instead of executing the \f[C]go build\f[R] command directly, you can 900 run it via the Makefile. 901 The default target changes the version suffix from \[dq]-DEV\[dq] to 902 \[dq]-beta\[dq] followed by additional commit details, embeds version 903 information binary resources on Windows, and copies the resulting rclone 904 executable into your GOPATH bin folder (\f[C]$(go env GOPATH)/bin\f[R], 905 which corresponds to \f[C]\[ti]/go/bin/rclone\f[R] by default). 906 .IP 907 .nf 908 \f[C] 909 make 910 \f[R] 911 .fi 912 .PP 913 To include mount command on macOS and Windows with Makefile build: 914 .IP 915 .nf 916 \f[C] 917 make GOTAGS=cmount 918 \f[R] 919 .fi 920 .PP 921 There are other make targets that can be used for more advanced builds, 922 such as cross-compiling for all supported os/architectures, and 923 packaging results into release artifacts. 924 See Makefile (https://github.com/artpar/artpar/blob/master/Makefile) and 925 cross-compile.go (https://github.com/artpar/artpar/blob/master/bin/cross-compile.go) 926 for details. 927 .PP 928 Another alternative method for source installation is to download the 929 source, build and install rclone - all in one operation, as a regular Go 930 package. 931 The source will be stored it in the Go module cache, and the resulting 932 executable will be in your GOPATH bin folder 933 (\f[C]$(go env GOPATH)/bin\f[R], which corresponds to 934 \f[C]\[ti]/go/bin/rclone\f[R] by default). 935 .IP 936 .nf 937 \f[C] 938 go install github.com/artpar/artpar\[at]latest 939 \f[R] 940 .fi 941 .SS Ansible installation 942 .PP 943 This can be done with Stefan Weichinger\[aq]s ansible 944 role (https://github.com/stefangweichinger/ansible-rclone). 945 .PP 946 Instructions 947 .IP "1." 3 948 \f[C]git clone https://github.com/stefangweichinger/ansible-rclone.git\f[R] 949 into your local roles-directory 950 .IP "2." 3 951 add the role to the hosts you want rclone installed to: 952 .IP 953 .nf 954 \f[C] 955 - hosts: rclone-hosts 956 roles: 957 - rclone 958 \f[R] 959 .fi 960 .SS Portable installation 961 .PP 962 As mentioned above (https://rclone.org/install/#quickstart), rclone is 963 single executable (\f[C]rclone\f[R], or \f[C]rclone.exe\f[R] on Windows) 964 that you can download as a zip archive and extract into a location of 965 your choosing. 966 When executing different commands, it may create files in different 967 locations, such as a configuration file and various temporary files. 968 By default the locations for these are according to your operating 969 system, e.g. 970 configuration file in your user profile directory and temporary files in 971 the standard temporary directory, but you can customize all of them, 972 e.g. 973 to make a completely self-contained, portable installation. 974 .PP 975 Run the config paths (https://rclone.org/commands/rclone_config_paths/) 976 command to see the locations that rclone will use. 977 .PP 978 To override them set the corresponding options (as command-line 979 arguments, or as environment 980 variables (https://rclone.org/docs/#environment-variables)): - 981 --config (https://rclone.org/docs/#config-config-file) - 982 --cache-dir (https://rclone.org/docs/#cache-dir-dir) - 983 --temp-dir (https://rclone.org/docs/#temp-dir-dir) 984 .SS Autostart 985 .PP 986 After installing and configuring rclone, as described above, you are 987 ready to use rclone as an interactive command line utility. 988 If your goal is to perform \f[I]periodic\f[R] operations, such as a 989 regular sync (https://rclone.org/commands/rclone_sync/), you will 990 probably want to configure your rclone command in your operating 991 system\[aq]s scheduler. 992 If you need to expose \f[I]service\f[R]-like features, such as remote 993 control (https://rclone.org/rc/), GUI (https://rclone.org/gui/), 994 serve (https://rclone.org/commands/rclone_serve/) or 995 mount (https://rclone.org/commands/rclone_mount/), you will often want 996 an rclone command always running in the background, and configuring it 997 to run in a service infrastructure may be a better option. 998 Below are some alternatives on how to achieve this on different 999 operating systems. 1000 .PP 1001 NOTE: Before setting up autorun it is highly recommended that you have 1002 tested your command manually from a Command Prompt first. 1003 .SS Autostart on Windows 1004 .PP 1005 The most relevant alternatives for autostart on Windows are: - Run at 1006 user log on using the Startup folder - Run at user log on, at system 1007 startup or at schedule using Task Scheduler - Run at system startup 1008 using Windows service 1009 .SS Running in background 1010 .PP 1011 Rclone is a console application, so if not starting from an existing 1012 Command Prompt, e.g. 1013 when starting rclone.exe from a shortcut, it will open a Command Prompt 1014 window. 1015 When configuring rclone to run from task scheduler and windows service 1016 you are able to set it to run hidden in background. 1017 From rclone version 1.54 you can also make it run hidden from anywhere 1018 by adding option \f[C]--no-console\f[R] (it may still flash briefly when 1019 the program starts). 1020 Since rclone normally writes information and any error messages to the 1021 console, you must redirect this to a file to be able to see it. 1022 Rclone has a built-in option \f[C]--log-file\f[R] for that. 1023 .PP 1024 Example command to run a sync in background: 1025 .IP 1026 .nf 1027 \f[C] 1028 c:\[rs]rclone\[rs]rclone.exe sync c:\[rs]files remote:/files --no-console --log-file c:\[rs]rclone\[rs]logs\[rs]sync_files.txt 1029 \f[R] 1030 .fi 1031 .SS User account 1032 .PP 1033 As mentioned in the mount (https://rclone.org/commands/rclone_mount/) 1034 documentation, mounted drives created as Administrator are not visible 1035 to other accounts, not even the account that was elevated as 1036 Administrator. 1037 By running the mount command as the built-in \f[C]SYSTEM\f[R] user 1038 account, it will create drives accessible for everyone on the system. 1039 Both scheduled task and Windows service can be used to achieve this. 1040 .PP 1041 NOTE: Remember that when rclone runs as the \f[C]SYSTEM\f[R] user, the 1042 user profile that it sees will not be yours. 1043 This means that if you normally run rclone with configuration file in 1044 the default location, to be able to use the same configuration when 1045 running as the system user you must explicitly tell rclone where to find 1046 it with the 1047 \f[C]--config\f[R] (https://rclone.org/docs/#config-config-file) option, 1048 or else it will look in the system users profile path 1049 (\f[C]C:\[rs]Windows\[rs]System32\[rs]config\[rs]systemprofile\f[R]). 1050 To test your command manually from a Command Prompt, you can run it with 1051 the 1052 PsExec (https://docs.microsoft.com/en-us/sysinternals/downloads/psexec) 1053 utility from Microsoft\[aq]s Sysinternals suite, which takes option 1054 \f[C]-s\f[R] to execute commands as the \f[C]SYSTEM\f[R] user. 1055 .SS Start from Startup folder 1056 .PP 1057 To quickly execute an rclone command you can simply create a standard 1058 Windows Explorer shortcut for the complete rclone command you want to 1059 run. 1060 If you store this shortcut in the special \[dq]Startup\[dq] start-menu 1061 folder, Windows will automatically run it at login. 1062 To open this folder in Windows Explorer, enter path 1063 \f[C]%APPDATA%\[rs]Microsoft\[rs]Windows\[rs]Start Menu\[rs]Programs\[rs]Startup\f[R], 1064 or 1065 \f[C]C:\[rs]ProgramData\[rs]Microsoft\[rs]Windows\[rs]Start Menu\[rs]Programs\[rs]StartUp\f[R] 1066 if you want the command to start for \f[I]every\f[R] user that logs in. 1067 .PP 1068 This is the easiest approach to autostarting of rclone, but it offers no 1069 functionality to set it to run as different user, or to set conditions 1070 or actions on certain events. 1071 Setting up a scheduled task as described below will often give you 1072 better results. 1073 .SS Start from Task Scheduler 1074 .PP 1075 Task Scheduler is an administrative tool built into Windows, and it can 1076 be used to configure rclone to be started automatically in a highly 1077 configurable way, e.g. 1078 periodically on a schedule, on user log on, or at system startup. 1079 It can run be configured to run as the current user, or for a mount 1080 command that needs to be available to all users it can run as the 1081 \f[C]SYSTEM\f[R] user. 1082 For technical information, see 1083 https://docs.microsoft.com/windows/win32/taskschd/task-scheduler-start-page. 1084 .SS Run as service 1085 .PP 1086 For running rclone at system startup, you can create a Windows service 1087 that executes your rclone command, as an alternative to scheduled task 1088 configured to run at startup. 1089 .SS Mount command built-in service integration 1090 .PP 1091 For mount commands, rclone has a built-in Windows service integration 1092 via the third-party WinFsp library it uses. 1093 Registering as a regular Windows service easy, as you just have to 1094 execute the built-in PowerShell command \f[C]New-Service\f[R] (requires 1095 administrative privileges). 1096 .PP 1097 Example of a PowerShell command that creates a Windows service for 1098 mounting some \f[C]remote:/files\f[R] as drive letter \f[C]X:\f[R], for 1099 \f[I]all\f[R] users (service will be running as the local system 1100 account): 1101 .IP 1102 .nf 1103 \f[C] 1104 New-Service -Name Rclone -BinaryPathName \[aq]c:\[rs]rclone\[rs]rclone.exe mount remote:/files X: --config c:\[rs]rclone\[rs]config\[rs]rclone.conf --log-file c:\[rs]rclone\[rs]logs\[rs]mount.txt\[aq] 1105 \f[R] 1106 .fi 1107 .PP 1108 The WinFsp service 1109 infrastructure (https://github.com/billziss-gh/winfsp/wiki/WinFsp-Service-Architecture) 1110 supports incorporating services for file system implementations, such as 1111 rclone, into its own launcher service, as kind of \[dq]child 1112 services\[dq]. 1113 This has the additional advantage that it also implements a network 1114 provider that integrates into Windows standard methods for managing 1115 network drives. 1116 This is currently not officially supported by Rclone, but with WinFsp 1117 version 2019.3 B2 / v1.5B2 or later it should be possible through path 1118 rewriting as described 1119 here (https://github.com/artpar/artpar/issues/3340). 1120 .SS Third-party service integration 1121 .PP 1122 To Windows service running any rclone command, the excellent third-party 1123 utility NSSM (http://nssm.cc), the \[dq]Non-Sucking Service 1124 Manager\[dq], can be used. 1125 It includes some advanced features such as adjusting process priority, 1126 defining process environment variables, redirect to file anything 1127 written to stdout, and customized response to different exit codes, with 1128 a GUI to configure everything from (although it can also be used from 1129 command line ). 1130 .PP 1131 There are also several other alternatives. 1132 To mention one more, WinSW (https://github.com/winsw/winsw), 1133 \[dq]Windows Service Wrapper\[dq], is worth checking out. 1134 It requires .NET Framework, but it is preinstalled on newer versions of 1135 Windows, and it also provides alternative standalone distributions which 1136 includes necessary runtime (.NET 5). 1137 WinSW is a command-line only utility, where you have to manually create 1138 an XML file with service configuration. 1139 This may be a drawback for some, but it can also be an advantage as it 1140 is easy to back up and reuse the configuration settings, without having 1141 go through manual steps in a GUI. 1142 One thing to note is that by default it does not restart the service on 1143 error, one have to explicit enable this in the configuration file (via 1144 the \[dq]onfailure\[dq] parameter). 1145 .SS Autostart on Linux 1146 .SS Start as a service 1147 .PP 1148 To always run rclone in background, relevant for mount commands etc, you 1149 can use systemd to set up rclone as a system or user service. 1150 Running as a system service ensures that it is run at startup even if 1151 the user it is running as has no active session. 1152 Running rclone as a user service ensures that it only starts after the 1153 configured user has logged into the system. 1154 .SS Run periodically from cron 1155 .PP 1156 To run a periodic command, such as a copy/sync, you can set up a cron 1157 job. 1158 .SH Usage 1159 .PP 1160 Rclone is a command line program to manage files on cloud storage. 1161 After download (https://rclone.org/downloads/) and install, continue 1162 here to learn how to use it: Initial configuration, what the basic 1163 syntax looks like, describes the various subcommands, the various 1164 options, and more. 1165 .SS Configure 1166 .PP 1167 First, you\[aq]ll need to configure rclone. 1168 As the object storage systems have quite complicated authentication 1169 these are kept in a config file. 1170 (See the \f[C]--config\f[R] entry for how to find the config file and 1171 choose its location.) 1172 .PP 1173 The easiest way to make the config is to run rclone with the config 1174 option: 1175 .IP 1176 .nf 1177 \f[C] 1178 rclone config 1179 \f[R] 1180 .fi 1181 .PP 1182 See the following for detailed instructions for 1183 .IP \[bu] 2 1184 1Fichier (https://rclone.org/fichier/) 1185 .IP \[bu] 2 1186 Akamai Netstorage (https://rclone.org/netstorage/) 1187 .IP \[bu] 2 1188 Alias (https://rclone.org/alias/) 1189 .IP \[bu] 2 1190 Amazon S3 (https://rclone.org/s3/) 1191 .IP \[bu] 2 1192 Backblaze B2 (https://rclone.org/b2/) 1193 .IP \[bu] 2 1194 Box (https://rclone.org/box/) 1195 .IP \[bu] 2 1196 Chunker (https://rclone.org/chunker/) - transparently splits large files 1197 for other remotes 1198 .IP \[bu] 2 1199 Citrix ShareFile (https://rclone.org/sharefile/) 1200 .IP \[bu] 2 1201 Compress (https://rclone.org/compress/) 1202 .IP \[bu] 2 1203 Combine (https://rclone.org/combine/) 1204 .IP \[bu] 2 1205 Crypt (https://rclone.org/crypt/) - to encrypt other remotes 1206 .IP \[bu] 2 1207 DigitalOcean Spaces (https://rclone.org/s3/#digitalocean-spaces) 1208 .IP \[bu] 2 1209 Digi Storage (https://rclone.org/koofr/#digi-storage) 1210 .IP \[bu] 2 1211 Dropbox (https://rclone.org/dropbox/) 1212 .IP \[bu] 2 1213 Enterprise File Fabric (https://rclone.org/filefabric/) 1214 .IP \[bu] 2 1215 FTP (https://rclone.org/ftp/) 1216 .IP \[bu] 2 1217 Google Cloud Storage (https://rclone.org/googlecloudstorage/) 1218 .IP \[bu] 2 1219 Google Drive (https://rclone.org/drive/) 1220 .IP \[bu] 2 1221 Google Photos (https://rclone.org/googlephotos/) 1222 .IP \[bu] 2 1223 Hasher (https://rclone.org/hasher/) - to handle checksums for other 1224 remotes 1225 .IP \[bu] 2 1226 HDFS (https://rclone.org/hdfs/) 1227 .IP \[bu] 2 1228 HiDrive (https://rclone.org/hidrive/) 1229 .IP \[bu] 2 1230 HTTP (https://rclone.org/http/) 1231 .IP \[bu] 2 1232 Internet Archive (https://rclone.org/internetarchive/) 1233 .IP \[bu] 2 1234 Jottacloud (https://rclone.org/jottacloud/) 1235 .IP \[bu] 2 1236 Koofr (https://rclone.org/koofr/) 1237 .IP \[bu] 2 1238 Linkbox (https://rclone.org/linkbox/) 1239 .IP \[bu] 2 1240 Mail.ru Cloud (https://rclone.org/mailru/) 1241 .IP \[bu] 2 1242 Mega (https://rclone.org/mega/) 1243 .IP \[bu] 2 1244 Memory (https://rclone.org/memory/) 1245 .IP \[bu] 2 1246 Microsoft Azure Blob Storage (https://rclone.org/azureblob/) 1247 .IP \[bu] 2 1248 Microsoft Azure Files Storage (https://rclone.org/azurefiles/) 1249 .IP \[bu] 2 1250 Microsoft OneDrive (https://rclone.org/onedrive/) 1251 .IP \[bu] 2 1252 OpenStack Swift / Rackspace Cloudfiles / Blomp Cloud Storage / Memset 1253 Memstore (https://rclone.org/swift/) 1254 .IP \[bu] 2 1255 OpenDrive (https://rclone.org/opendrive/) 1256 .IP \[bu] 2 1257 Oracle Object Storage (https://rclone.org/oracleobjectstorage/) 1258 .IP \[bu] 2 1259 Pcloud (https://rclone.org/pcloud/) 1260 .IP \[bu] 2 1261 PikPak (https://rclone.org/pikpak/) 1262 .IP \[bu] 2 1263 premiumize.me (https://rclone.org/premiumizeme/) 1264 .IP \[bu] 2 1265 put.io (https://rclone.org/putio/) 1266 .IP \[bu] 2 1267 Proton Drive (https://rclone.org/protondrive/) 1268 .IP \[bu] 2 1269 QingStor (https://rclone.org/qingstor/) 1270 .IP \[bu] 2 1271 Quatrix by Maytech (https://rclone.org/quatrix/) 1272 .IP \[bu] 2 1273 Seafile (https://rclone.org/seafile/) 1274 .IP \[bu] 2 1275 SFTP (https://rclone.org/sftp/) 1276 .IP \[bu] 2 1277 Sia (https://rclone.org/sia/) 1278 .IP \[bu] 2 1279 SMB (https://rclone.org/smb/) 1280 .IP \[bu] 2 1281 Storj (https://rclone.org/storj/) 1282 .IP \[bu] 2 1283 SugarSync (https://rclone.org/sugarsync/) 1284 .IP \[bu] 2 1285 Union (https://rclone.org/union/) 1286 .IP \[bu] 2 1287 Uptobox (https://rclone.org/uptobox/) 1288 .IP \[bu] 2 1289 WebDAV (https://rclone.org/webdav/) 1290 .IP \[bu] 2 1291 Yandex Disk (https://rclone.org/yandex/) 1292 .IP \[bu] 2 1293 Zoho WorkDrive (https://rclone.org/zoho/) 1294 .IP \[bu] 2 1295 The local filesystem (https://rclone.org/local/) 1296 .SS Basic syntax 1297 .PP 1298 Rclone syncs a directory tree from one storage system to another. 1299 .PP 1300 Its syntax is like this 1301 .IP 1302 .nf 1303 \f[C] 1304 Syntax: [options] subcommand <parameters> <parameters...> 1305 \f[R] 1306 .fi 1307 .PP 1308 Source and destination paths are specified by the name you gave the 1309 storage system in the config file then the sub path, e.g. 1310 \[dq]drive:myfolder\[dq] to look at \[dq]myfolder\[dq] in Google drive. 1311 .PP 1312 You can define as many storage paths as you like in the config file. 1313 .PP 1314 Please use the \f[C]--interactive\f[R]/\f[C]-i\f[R] flag while learning 1315 rclone to avoid accidental data loss. 1316 .SS Subcommands 1317 .PP 1318 rclone uses a system of subcommands. 1319 For example 1320 .IP 1321 .nf 1322 \f[C] 1323 rclone ls remote:path # lists a remote 1324 rclone copy /local/path remote:path # copies /local/path to the remote 1325 rclone sync --interactive /local/path remote:path # syncs /local/path to the remote 1326 \f[R] 1327 .fi 1328 .SH rclone config 1329 .PP 1330 Enter an interactive configuration session. 1331 .SS Synopsis 1332 .PP 1333 Enter an interactive configuration session where you can setup new 1334 remotes and manage existing ones. 1335 You may also set or remove a password to protect your configuration. 1336 .IP 1337 .nf 1338 \f[C] 1339 rclone config [flags] 1340 \f[R] 1341 .fi 1342 .SS Options 1343 .IP 1344 .nf 1345 \f[C] 1346 -h, --help help for config 1347 \f[R] 1348 .fi 1349 .PP 1350 See the global flags page (https://rclone.org/flags/) for global options 1351 not listed here. 1352 .SH SEE ALSO 1353 .IP \[bu] 2 1354 rclone (https://rclone.org/commands/rclone/) - Show help for rclone 1355 commands, flags and backends. 1356 .IP \[bu] 2 1357 rclone config create (https://rclone.org/commands/rclone_config_create/) 1358 - Create a new remote with name, type and options. 1359 .IP \[bu] 2 1360 rclone config delete (https://rclone.org/commands/rclone_config_delete/) 1361 - Delete an existing remote. 1362 .IP \[bu] 2 1363 rclone config 1364 disconnect (https://rclone.org/commands/rclone_config_disconnect/) - 1365 Disconnects user from remote 1366 .IP \[bu] 2 1367 rclone config dump (https://rclone.org/commands/rclone_config_dump/) - 1368 Dump the config file as JSON. 1369 .IP \[bu] 2 1370 rclone config edit (https://rclone.org/commands/rclone_config_edit/) - 1371 Enter an interactive configuration session. 1372 .IP \[bu] 2 1373 rclone config file (https://rclone.org/commands/rclone_config_file/) - 1374 Show path of configuration file in use. 1375 .IP \[bu] 2 1376 rclone config 1377 password (https://rclone.org/commands/rclone_config_password/) - Update 1378 password in an existing remote. 1379 .IP \[bu] 2 1380 rclone config paths (https://rclone.org/commands/rclone_config_paths/) - 1381 Show paths used for configuration, cache, temp etc. 1382 .IP \[bu] 2 1383 rclone config 1384 providers (https://rclone.org/commands/rclone_config_providers/) - List 1385 in JSON format all the providers and options. 1386 .IP \[bu] 2 1387 rclone config 1388 reconnect (https://rclone.org/commands/rclone_config_reconnect/) - 1389 Re-authenticates user with remote. 1390 .IP \[bu] 2 1391 rclone config 1392 redacted (https://rclone.org/commands/rclone_config_redacted/) - Print 1393 redacted (decrypted) config file, or the redacted config for a single 1394 remote. 1395 .IP \[bu] 2 1396 rclone config show (https://rclone.org/commands/rclone_config_show/) - 1397 Print (decrypted) config file, or the config for a single remote. 1398 .IP \[bu] 2 1399 rclone config touch (https://rclone.org/commands/rclone_config_touch/) - 1400 Ensure configuration file exists. 1401 .IP \[bu] 2 1402 rclone config update (https://rclone.org/commands/rclone_config_update/) 1403 - Update options in an existing remote. 1404 .IP \[bu] 2 1405 rclone config 1406 userinfo (https://rclone.org/commands/rclone_config_userinfo/) - Prints 1407 info about logged in user of remote. 1408 .SH rclone copy 1409 .PP 1410 Copy files from source to dest, skipping identical files. 1411 .SS Synopsis 1412 .PP 1413 Copy the source to the destination. 1414 Does not transfer files that are identical on source and destination, 1415 testing by size and modification time or MD5SUM. 1416 Doesn\[aq]t delete files from the destination. 1417 If you want to also delete files from destination, to make it match 1418 source, use the sync (https://rclone.org/commands/rclone_sync/) command 1419 instead. 1420 .PP 1421 Note that it is always the contents of the directory that is synced, not 1422 the directory itself. 1423 So when source:path is a directory, it\[aq]s the contents of source:path 1424 that are copied, not the directory name and contents. 1425 .PP 1426 To copy single files, use the 1427 copyto (https://rclone.org/commands/rclone_copyto/) command instead. 1428 .PP 1429 If dest:path doesn\[aq]t exist, it is created and the source:path 1430 contents go there. 1431 .PP 1432 For example 1433 .IP 1434 .nf 1435 \f[C] 1436 rclone copy source:sourcepath dest:destpath 1437 \f[R] 1438 .fi 1439 .PP 1440 Let\[aq]s say there are two files in sourcepath 1441 .IP 1442 .nf 1443 \f[C] 1444 sourcepath/one.txt 1445 sourcepath/two.txt 1446 \f[R] 1447 .fi 1448 .PP 1449 This copies them to 1450 .IP 1451 .nf 1452 \f[C] 1453 destpath/one.txt 1454 destpath/two.txt 1455 \f[R] 1456 .fi 1457 .PP 1458 Not to 1459 .IP 1460 .nf 1461 \f[C] 1462 destpath/sourcepath/one.txt 1463 destpath/sourcepath/two.txt 1464 \f[R] 1465 .fi 1466 .PP 1467 If you are familiar with \f[C]rsync\f[R], rclone always works as if you 1468 had written a trailing \f[C]/\f[R] - meaning \[dq]copy the contents of 1469 this directory\[dq]. 1470 This applies to all commands and whether you are talking about the 1471 source or destination. 1472 .PP 1473 See the --no-traverse (https://rclone.org/docs/#no-traverse) option for 1474 controlling whether rclone lists the destination directory or not. 1475 Supplying this option when copying a small number of files into a large 1476 destination can speed transfers up greatly. 1477 .PP 1478 For example, if you have many files in /path/to/src but only a few of 1479 them change every day, you can copy all the files which have changed 1480 recently very efficiently like this: 1481 .IP 1482 .nf 1483 \f[C] 1484 rclone copy --max-age 24h --no-traverse /path/to/src remote: 1485 \f[R] 1486 .fi 1487 .PP 1488 Rclone will sync the modification times of files and directories if the 1489 backend supports it. 1490 If metadata syncing is required then use the \f[C]--metadata\f[R] flag. 1491 .PP 1492 Note that the modification time and metadata for the root directory will 1493 \f[B]not\f[R] be synced. 1494 See https://github.com/artpar/artpar/issues/7652 for more info. 1495 .PP 1496 \f[B]Note\f[R]: Use the \f[C]-P\f[R]/\f[C]--progress\f[R] flag to view 1497 real-time transfer statistics. 1498 .PP 1499 \f[B]Note\f[R]: Use the \f[C]--dry-run\f[R] or the 1500 \f[C]--interactive\f[R]/\f[C]-i\f[R] flag to test without copying 1501 anything. 1502 .IP 1503 .nf 1504 \f[C] 1505 rclone copy source:path dest:path [flags] 1506 \f[R] 1507 .fi 1508 .SS Options 1509 .IP 1510 .nf 1511 \f[C] 1512 --create-empty-src-dirs Create empty source dirs on destination after copy 1513 -h, --help help for copy 1514 \f[R] 1515 .fi 1516 .SS Copy Options 1517 .PP 1518 Flags for anything which can Copy a file. 1519 .IP 1520 .nf 1521 \f[C] 1522 --check-first Do all the checks before starting transfers 1523 -c, --checksum Check for changes with size & checksum (if available, or fallback to size only). 1524 --compare-dest stringArray Include additional comma separated server-side paths during comparison 1525 --copy-dest stringArray Implies --compare-dest but also copies files from paths into destination 1526 --cutoff-mode HARD|SOFT|CAUTIOUS Mode to stop transfers when reaching the max transfer limit HARD|SOFT|CAUTIOUS (default HARD) 1527 --ignore-case-sync Ignore case when synchronizing 1528 --ignore-checksum Skip post copy check of checksums 1529 --ignore-existing Skip all files that exist on destination 1530 --ignore-size Ignore size when skipping use modtime or checksum 1531 -I, --ignore-times Don\[aq]t skip items that match size and time - transfer all unconditionally 1532 --immutable Do not modify files, fail if existing files have been modified 1533 --inplace Download directly to destination file instead of atomic download to temp/rename 1534 --max-backlog int Maximum number of objects in sync or check backlog (default 10000) 1535 --max-duration Duration Maximum duration rclone will transfer data for (default 0s) 1536 --max-transfer SizeSuffix Maximum size of data to transfer (default off) 1537 -M, --metadata If set, preserve metadata when copying objects 1538 --modify-window Duration Max time diff to be considered the same (default 1ns) 1539 --multi-thread-chunk-size SizeSuffix Chunk size for multi-thread downloads / uploads, if not set by filesystem (default 64Mi) 1540 --multi-thread-cutoff SizeSuffix Use multi-thread downloads for files above this size (default 256Mi) 1541 --multi-thread-streams int Number of streams to use for multi-thread downloads (default 4) 1542 --multi-thread-write-buffer-size SizeSuffix In memory buffer size for writing when in multi-thread mode (default 128Ki) 1543 --no-check-dest Don\[aq]t check the destination, copy regardless 1544 --no-traverse Don\[aq]t traverse destination file system on copy 1545 --no-update-dir-modtime Don\[aq]t update directory modification times 1546 --no-update-modtime Don\[aq]t update destination modtime if files identical 1547 --order-by string Instructions on how to order the transfers, e.g. \[aq]size,descending\[aq] 1548 --partial-suffix string Add partial-suffix to temporary file name when --inplace is not used (default \[dq].partial\[dq]) 1549 --refresh-times Refresh the modtime of remote files 1550 --server-side-across-configs Allow server-side operations (e.g. copy) to work across different configs 1551 --size-only Skip based on size only, not modtime or checksum 1552 --streaming-upload-cutoff SizeSuffix Cutoff for switching to chunked upload if file size is unknown, upload starts after reaching cutoff or when file ends (default 100Ki) 1553 -u, --update Skip files that are newer on the destination 1554 \f[R] 1555 .fi 1556 .SS Important Options 1557 .PP 1558 Important flags useful for most commands. 1559 .IP 1560 .nf 1561 \f[C] 1562 -n, --dry-run Do a trial run with no permanent changes 1563 -i, --interactive Enable interactive mode 1564 -v, --verbose count Print lots more stuff (repeat for more) 1565 \f[R] 1566 .fi 1567 .SS Filter Options 1568 .PP 1569 Flags for filtering directory listings. 1570 .IP 1571 .nf 1572 \f[C] 1573 --delete-excluded Delete files on dest excluded from sync 1574 --exclude stringArray Exclude files matching pattern 1575 --exclude-from stringArray Read file exclude patterns from file (use - to read from stdin) 1576 --exclude-if-present stringArray Exclude directories if filename is present 1577 --files-from stringArray Read list of source-file names from file (use - to read from stdin) 1578 --files-from-raw stringArray Read list of source-file names from file without any processing of lines (use - to read from stdin) 1579 -f, --filter stringArray Add a file filtering rule 1580 --filter-from stringArray Read file filtering patterns from a file (use - to read from stdin) 1581 --ignore-case Ignore case in filters (case insensitive) 1582 --include stringArray Include files matching pattern 1583 --include-from stringArray Read file include patterns from file (use - to read from stdin) 1584 --max-age Duration Only transfer files younger than this in s or suffix ms|s|m|h|d|w|M|y (default off) 1585 --max-depth int If set limits the recursion depth to this (default -1) 1586 --max-size SizeSuffix Only transfer files smaller than this in KiB or suffix B|K|M|G|T|P (default off) 1587 --metadata-exclude stringArray Exclude metadatas matching pattern 1588 --metadata-exclude-from stringArray Read metadata exclude patterns from file (use - to read from stdin) 1589 --metadata-filter stringArray Add a metadata filtering rule 1590 --metadata-filter-from stringArray Read metadata filtering patterns from a file (use - to read from stdin) 1591 --metadata-include stringArray Include metadatas matching pattern 1592 --metadata-include-from stringArray Read metadata include patterns from file (use - to read from stdin) 1593 --min-age Duration Only transfer files older than this in s or suffix ms|s|m|h|d|w|M|y (default off) 1594 --min-size SizeSuffix Only transfer files bigger than this in KiB or suffix B|K|M|G|T|P (default off) 1595 \f[R] 1596 .fi 1597 .SS Listing Options 1598 .PP 1599 Flags for listing directories. 1600 .IP 1601 .nf 1602 \f[C] 1603 --default-time Time Time to show if modtime is unknown for files and directories (default 2000-01-01T00:00:00Z) 1604 --fast-list Use recursive list if available; uses more memory but fewer transactions 1605 \f[R] 1606 .fi 1607 .PP 1608 See the global flags page (https://rclone.org/flags/) for global options 1609 not listed here. 1610 .SH SEE ALSO 1611 .IP \[bu] 2 1612 rclone (https://rclone.org/commands/rclone/) - Show help for rclone 1613 commands, flags and backends. 1614 .SH rclone sync 1615 .PP 1616 Make source and dest identical, modifying destination only. 1617 .SS Synopsis 1618 .PP 1619 Sync the source to the destination, changing the destination only. 1620 Doesn\[aq]t transfer files that are identical on source and destination, 1621 testing by size and modification time or MD5SUM. 1622 Destination is updated to match source, including deleting files if 1623 necessary (except duplicate objects, see below). 1624 If you don\[aq]t want to delete files from destination, use the 1625 copy (https://rclone.org/commands/rclone_copy/) command instead. 1626 .PP 1627 \f[B]Important\f[R]: Since this can cause data loss, test first with the 1628 \f[C]--dry-run\f[R] or the \f[C]--interactive\f[R]/\f[C]-i\f[R] flag. 1629 .IP 1630 .nf 1631 \f[C] 1632 rclone sync --interactive SOURCE remote:DESTINATION 1633 \f[R] 1634 .fi 1635 .PP 1636 Note that files in the destination won\[aq]t be deleted if there were 1637 any errors at any point. 1638 Duplicate objects (files with the same name, on those providers that 1639 support it) are also not yet handled. 1640 .PP 1641 It is always the contents of the directory that is synced, not the 1642 directory itself. 1643 So when source:path is a directory, it\[aq]s the contents of source:path 1644 that are copied, not the directory name and contents. 1645 See extended explanation in the 1646 copy (https://rclone.org/commands/rclone_copy/) command if unsure. 1647 .PP 1648 If dest:path doesn\[aq]t exist, it is created and the source:path 1649 contents go there. 1650 .PP 1651 It is not possible to sync overlapping remotes. 1652 However, you may exclude the destination from the sync with a filter 1653 rule or by putting an exclude-if-present file inside the destination 1654 directory and sync to a destination that is inside the source directory. 1655 .PP 1656 Rclone will sync the modification times of files and directories if the 1657 backend supports it. 1658 If metadata syncing is required then use the \f[C]--metadata\f[R] flag. 1659 .PP 1660 Note that the modification time and metadata for the root directory will 1661 \f[B]not\f[R] be synced. 1662 See https://github.com/artpar/artpar/issues/7652 for more info. 1663 .PP 1664 \f[B]Note\f[R]: Use the \f[C]-P\f[R]/\f[C]--progress\f[R] flag to view 1665 real-time transfer statistics 1666 .PP 1667 \f[B]Note\f[R]: Use the \f[C]rclone dedupe\f[R] command to deal with 1668 \[dq]Duplicate object/directory found in source/destination - 1669 ignoring\[dq] errors. 1670 See this forum 1671 post (https://forum.rclone.org/t/sync-not-clearing-duplicates/14372) for 1672 more info. 1673 .SH Logger Flags 1674 .PP 1675 The \f[C]--differ\f[R], \f[C]--missing-on-dst\f[R], 1676 \f[C]--missing-on-src\f[R], \f[C]--match\f[R] and \f[C]--error\f[R] 1677 flags write paths, one per line, to the file name (or stdout if it is 1678 \f[C]-\f[R]) supplied. 1679 What they write is described in the help below. 1680 For example \f[C]--differ\f[R] will write all paths which are present on 1681 both the source and destination but different. 1682 .PP 1683 The \f[C]--combined\f[R] flag will write a file (or stdout) which 1684 contains all file paths with a symbol and then a space and then the path 1685 to tell you what happened to it. 1686 These are reminiscent of diff files. 1687 .IP \[bu] 2 1688 \f[C]= path\f[R] means path was found in source and destination and was 1689 identical 1690 .IP \[bu] 2 1691 \[ga]- path\[ga] means path was missing on the source, so only in the 1692 destination 1693 .IP \[bu] 2 1694 \[ga]+ path\[ga] means path was missing on the destination, so only in 1695 the source 1696 .IP \[bu] 2 1697 \[ga]* path\[ga] means path was present in source and destination but 1698 different. 1699 .IP \[bu] 2 1700 \f[C]! path\f[R] means there was an error reading or hashing the source 1701 or dest. 1702 .PP 1703 The \f[C]--dest-after\f[R] flag writes a list file using the same format 1704 flags as 1705 \f[C]lsf\f[R] (https://rclone.org/commands/rclone_lsf/#synopsis) 1706 (including customizable options for hash, modtime, 1707 etc. (https://rclone.org/commands/rclone_lsf/#synopsis)) Conceptually it 1708 is similar to rsync\[aq]s \f[C]--itemize-changes\f[R], but not identical 1709 -- it should output an accurate list of what will be on the destination 1710 after the sync. 1711 .PP 1712 Note that these logger flags have a few limitations, and certain 1713 scenarios are not currently supported: 1714 .IP \[bu] 2 1715 \f[C]--max-duration\f[R] / \f[C]CutoffModeHard\f[R] 1716 .IP \[bu] 2 1717 \f[C]--compare-dest\f[R] / \f[C]--copy-dest\f[R] 1718 .IP \[bu] 2 1719 server-side moves of an entire dir at once 1720 .IP \[bu] 2 1721 High-level retries, because there would be duplicates (use 1722 \f[C]--retries 1\f[R] to disable) 1723 .IP \[bu] 2 1724 Possibly some unusual error scenarios 1725 .PP 1726 Note also that each file is logged during the sync, as opposed to after, 1727 so it is most useful as a predictor of what SHOULD happen to each file 1728 (which may or may not match what actually DID.) 1729 .IP 1730 .nf 1731 \f[C] 1732 rclone sync source:path dest:path [flags] 1733 \f[R] 1734 .fi 1735 .SS Options 1736 .IP 1737 .nf 1738 \f[C] 1739 --absolute Put a leading / in front of path names 1740 --combined string Make a combined report of changes to this file 1741 --create-empty-src-dirs Create empty source dirs on destination after sync 1742 --csv Output in CSV format 1743 --dest-after string Report all files that exist on the dest post-sync 1744 --differ string Report all non-matching files to this file 1745 -d, --dir-slash Append a slash to directory names (default true) 1746 --dirs-only Only list directories 1747 --error string Report all files with errors (hashing or reading) to this file 1748 --files-only Only list files (default true) 1749 -F, --format string Output format - see lsf help for details (default \[dq]p\[dq]) 1750 --hash h Use this hash when h is used in the format MD5|SHA-1|DropboxHash (default \[dq]md5\[dq]) 1751 -h, --help help for sync 1752 --match string Report all matching files to this file 1753 --missing-on-dst string Report all files missing from the destination to this file 1754 --missing-on-src string Report all files missing from the source to this file 1755 -s, --separator string Separator for the items in the format (default \[dq];\[dq]) 1756 -t, --timeformat string Specify a custom time format, or \[aq]max\[aq] for max precision supported by remote (default: 2006-01-02 15:04:05) 1757 \f[R] 1758 .fi 1759 .SS Copy Options 1760 .PP 1761 Flags for anything which can Copy a file. 1762 .IP 1763 .nf 1764 \f[C] 1765 --check-first Do all the checks before starting transfers 1766 -c, --checksum Check for changes with size & checksum (if available, or fallback to size only). 1767 --compare-dest stringArray Include additional comma separated server-side paths during comparison 1768 --copy-dest stringArray Implies --compare-dest but also copies files from paths into destination 1769 --cutoff-mode HARD|SOFT|CAUTIOUS Mode to stop transfers when reaching the max transfer limit HARD|SOFT|CAUTIOUS (default HARD) 1770 --ignore-case-sync Ignore case when synchronizing 1771 --ignore-checksum Skip post copy check of checksums 1772 --ignore-existing Skip all files that exist on destination 1773 --ignore-size Ignore size when skipping use modtime or checksum 1774 -I, --ignore-times Don\[aq]t skip items that match size and time - transfer all unconditionally 1775 --immutable Do not modify files, fail if existing files have been modified 1776 --inplace Download directly to destination file instead of atomic download to temp/rename 1777 --max-backlog int Maximum number of objects in sync or check backlog (default 10000) 1778 --max-duration Duration Maximum duration rclone will transfer data for (default 0s) 1779 --max-transfer SizeSuffix Maximum size of data to transfer (default off) 1780 -M, --metadata If set, preserve metadata when copying objects 1781 --modify-window Duration Max time diff to be considered the same (default 1ns) 1782 --multi-thread-chunk-size SizeSuffix Chunk size for multi-thread downloads / uploads, if not set by filesystem (default 64Mi) 1783 --multi-thread-cutoff SizeSuffix Use multi-thread downloads for files above this size (default 256Mi) 1784 --multi-thread-streams int Number of streams to use for multi-thread downloads (default 4) 1785 --multi-thread-write-buffer-size SizeSuffix In memory buffer size for writing when in multi-thread mode (default 128Ki) 1786 --no-check-dest Don\[aq]t check the destination, copy regardless 1787 --no-traverse Don\[aq]t traverse destination file system on copy 1788 --no-update-dir-modtime Don\[aq]t update directory modification times 1789 --no-update-modtime Don\[aq]t update destination modtime if files identical 1790 --order-by string Instructions on how to order the transfers, e.g. \[aq]size,descending\[aq] 1791 --partial-suffix string Add partial-suffix to temporary file name when --inplace is not used (default \[dq].partial\[dq]) 1792 --refresh-times Refresh the modtime of remote files 1793 --server-side-across-configs Allow server-side operations (e.g. copy) to work across different configs 1794 --size-only Skip based on size only, not modtime or checksum 1795 --streaming-upload-cutoff SizeSuffix Cutoff for switching to chunked upload if file size is unknown, upload starts after reaching cutoff or when file ends (default 100Ki) 1796 -u, --update Skip files that are newer on the destination 1797 \f[R] 1798 .fi 1799 .SS Sync Options 1800 .PP 1801 Flags just used for \f[C]rclone sync\f[R]. 1802 .IP 1803 .nf 1804 \f[C] 1805 --backup-dir string Make backups into hierarchy based in DIR 1806 --delete-after When synchronizing, delete files on destination after transferring (default) 1807 --delete-before When synchronizing, delete files on destination before transferring 1808 --delete-during When synchronizing, delete files during transfer 1809 --fix-case Force rename of case insensitive dest to match source 1810 --ignore-errors Delete even if there are I/O errors 1811 --max-delete int When synchronizing, limit the number of deletes (default -1) 1812 --max-delete-size SizeSuffix When synchronizing, limit the total size of deletes (default off) 1813 --suffix string Suffix to add to changed files 1814 --suffix-keep-extension Preserve the extension when using --suffix 1815 --track-renames When synchronizing, track file renames and do a server-side move if possible 1816 --track-renames-strategy string Strategies to use when synchronizing using track-renames hash|modtime|leaf (default \[dq]hash\[dq]) 1817 \f[R] 1818 .fi 1819 .SS Important Options 1820 .PP 1821 Important flags useful for most commands. 1822 .IP 1823 .nf 1824 \f[C] 1825 -n, --dry-run Do a trial run with no permanent changes 1826 -i, --interactive Enable interactive mode 1827 -v, --verbose count Print lots more stuff (repeat for more) 1828 \f[R] 1829 .fi 1830 .SS Filter Options 1831 .PP 1832 Flags for filtering directory listings. 1833 .IP 1834 .nf 1835 \f[C] 1836 --delete-excluded Delete files on dest excluded from sync 1837 --exclude stringArray Exclude files matching pattern 1838 --exclude-from stringArray Read file exclude patterns from file (use - to read from stdin) 1839 --exclude-if-present stringArray Exclude directories if filename is present 1840 --files-from stringArray Read list of source-file names from file (use - to read from stdin) 1841 --files-from-raw stringArray Read list of source-file names from file without any processing of lines (use - to read from stdin) 1842 -f, --filter stringArray Add a file filtering rule 1843 --filter-from stringArray Read file filtering patterns from a file (use - to read from stdin) 1844 --ignore-case Ignore case in filters (case insensitive) 1845 --include stringArray Include files matching pattern 1846 --include-from stringArray Read file include patterns from file (use - to read from stdin) 1847 --max-age Duration Only transfer files younger than this in s or suffix ms|s|m|h|d|w|M|y (default off) 1848 --max-depth int If set limits the recursion depth to this (default -1) 1849 --max-size SizeSuffix Only transfer files smaller than this in KiB or suffix B|K|M|G|T|P (default off) 1850 --metadata-exclude stringArray Exclude metadatas matching pattern 1851 --metadata-exclude-from stringArray Read metadata exclude patterns from file (use - to read from stdin) 1852 --metadata-filter stringArray Add a metadata filtering rule 1853 --metadata-filter-from stringArray Read metadata filtering patterns from a file (use - to read from stdin) 1854 --metadata-include stringArray Include metadatas matching pattern 1855 --metadata-include-from stringArray Read metadata include patterns from file (use - to read from stdin) 1856 --min-age Duration Only transfer files older than this in s or suffix ms|s|m|h|d|w|M|y (default off) 1857 --min-size SizeSuffix Only transfer files bigger than this in KiB or suffix B|K|M|G|T|P (default off) 1858 \f[R] 1859 .fi 1860 .SS Listing Options 1861 .PP 1862 Flags for listing directories. 1863 .IP 1864 .nf 1865 \f[C] 1866 --default-time Time Time to show if modtime is unknown for files and directories (default 2000-01-01T00:00:00Z) 1867 --fast-list Use recursive list if available; uses more memory but fewer transactions 1868 \f[R] 1869 .fi 1870 .PP 1871 See the global flags page (https://rclone.org/flags/) for global options 1872 not listed here. 1873 .SH SEE ALSO 1874 .IP \[bu] 2 1875 rclone (https://rclone.org/commands/rclone/) - Show help for rclone 1876 commands, flags and backends. 1877 .SH rclone move 1878 .PP 1879 Move files from source to dest. 1880 .SS Synopsis 1881 .PP 1882 Moves the contents of the source directory to the destination directory. 1883 Rclone will error if the source and destination overlap and the remote 1884 does not support a server-side directory move operation. 1885 .PP 1886 To move single files, use the 1887 moveto (https://rclone.org/commands/rclone_moveto/) command instead. 1888 .PP 1889 If no filters are in use and if possible this will server-side move 1890 \f[C]source:path\f[R] into \f[C]dest:path\f[R]. 1891 After this \f[C]source:path\f[R] will no longer exist. 1892 .PP 1893 Otherwise for each file in \f[C]source:path\f[R] selected by the filters 1894 (if any) this will move it into \f[C]dest:path\f[R]. 1895 If possible a server-side move will be used, otherwise it will copy it 1896 (server-side if possible) into \f[C]dest:path\f[R] then delete the 1897 original (if no errors on copy) in \f[C]source:path\f[R]. 1898 .PP 1899 If you want to delete empty source directories after move, use the 1900 \f[C]--delete-empty-src-dirs\f[R] flag. 1901 .PP 1902 See the --no-traverse (https://rclone.org/docs/#no-traverse) option for 1903 controlling whether rclone lists the destination directory or not. 1904 Supplying this option when moving a small number of files into a large 1905 destination can speed transfers up greatly. 1906 .PP 1907 Rclone will sync the modification times of files and directories if the 1908 backend supports it. 1909 If metadata syncing is required then use the \f[C]--metadata\f[R] flag. 1910 .PP 1911 Note that the modification time and metadata for the root directory will 1912 \f[B]not\f[R] be synced. 1913 See https://github.com/artpar/artpar/issues/7652 for more info. 1914 .PP 1915 \f[B]Important\f[R]: Since this can cause data loss, test first with the 1916 \f[C]--dry-run\f[R] or the \f[C]--interactive\f[R]/\f[C]-i\f[R] flag. 1917 .PP 1918 \f[B]Note\f[R]: Use the \f[C]-P\f[R]/\f[C]--progress\f[R] flag to view 1919 real-time transfer statistics. 1920 .IP 1921 .nf 1922 \f[C] 1923 rclone move source:path dest:path [flags] 1924 \f[R] 1925 .fi 1926 .SS Options 1927 .IP 1928 .nf 1929 \f[C] 1930 --create-empty-src-dirs Create empty source dirs on destination after move 1931 --delete-empty-src-dirs Delete empty source dirs after move 1932 -h, --help help for move 1933 \f[R] 1934 .fi 1935 .SS Copy Options 1936 .PP 1937 Flags for anything which can Copy a file. 1938 .IP 1939 .nf 1940 \f[C] 1941 --check-first Do all the checks before starting transfers 1942 -c, --checksum Check for changes with size & checksum (if available, or fallback to size only). 1943 --compare-dest stringArray Include additional comma separated server-side paths during comparison 1944 --copy-dest stringArray Implies --compare-dest but also copies files from paths into destination 1945 --cutoff-mode HARD|SOFT|CAUTIOUS Mode to stop transfers when reaching the max transfer limit HARD|SOFT|CAUTIOUS (default HARD) 1946 --ignore-case-sync Ignore case when synchronizing 1947 --ignore-checksum Skip post copy check of checksums 1948 --ignore-existing Skip all files that exist on destination 1949 --ignore-size Ignore size when skipping use modtime or checksum 1950 -I, --ignore-times Don\[aq]t skip items that match size and time - transfer all unconditionally 1951 --immutable Do not modify files, fail if existing files have been modified 1952 --inplace Download directly to destination file instead of atomic download to temp/rename 1953 --max-backlog int Maximum number of objects in sync or check backlog (default 10000) 1954 --max-duration Duration Maximum duration rclone will transfer data for (default 0s) 1955 --max-transfer SizeSuffix Maximum size of data to transfer (default off) 1956 -M, --metadata If set, preserve metadata when copying objects 1957 --modify-window Duration Max time diff to be considered the same (default 1ns) 1958 --multi-thread-chunk-size SizeSuffix Chunk size for multi-thread downloads / uploads, if not set by filesystem (default 64Mi) 1959 --multi-thread-cutoff SizeSuffix Use multi-thread downloads for files above this size (default 256Mi) 1960 --multi-thread-streams int Number of streams to use for multi-thread downloads (default 4) 1961 --multi-thread-write-buffer-size SizeSuffix In memory buffer size for writing when in multi-thread mode (default 128Ki) 1962 --no-check-dest Don\[aq]t check the destination, copy regardless 1963 --no-traverse Don\[aq]t traverse destination file system on copy 1964 --no-update-dir-modtime Don\[aq]t update directory modification times 1965 --no-update-modtime Don\[aq]t update destination modtime if files identical 1966 --order-by string Instructions on how to order the transfers, e.g. \[aq]size,descending\[aq] 1967 --partial-suffix string Add partial-suffix to temporary file name when --inplace is not used (default \[dq].partial\[dq]) 1968 --refresh-times Refresh the modtime of remote files 1969 --server-side-across-configs Allow server-side operations (e.g. copy) to work across different configs 1970 --size-only Skip based on size only, not modtime or checksum 1971 --streaming-upload-cutoff SizeSuffix Cutoff for switching to chunked upload if file size is unknown, upload starts after reaching cutoff or when file ends (default 100Ki) 1972 -u, --update Skip files that are newer on the destination 1973 \f[R] 1974 .fi 1975 .SS Important Options 1976 .PP 1977 Important flags useful for most commands. 1978 .IP 1979 .nf 1980 \f[C] 1981 -n, --dry-run Do a trial run with no permanent changes 1982 -i, --interactive Enable interactive mode 1983 -v, --verbose count Print lots more stuff (repeat for more) 1984 \f[R] 1985 .fi 1986 .SS Filter Options 1987 .PP 1988 Flags for filtering directory listings. 1989 .IP 1990 .nf 1991 \f[C] 1992 --delete-excluded Delete files on dest excluded from sync 1993 --exclude stringArray Exclude files matching pattern 1994 --exclude-from stringArray Read file exclude patterns from file (use - to read from stdin) 1995 --exclude-if-present stringArray Exclude directories if filename is present 1996 --files-from stringArray Read list of source-file names from file (use - to read from stdin) 1997 --files-from-raw stringArray Read list of source-file names from file without any processing of lines (use - to read from stdin) 1998 -f, --filter stringArray Add a file filtering rule 1999 --filter-from stringArray Read file filtering patterns from a file (use - to read from stdin) 2000 --ignore-case Ignore case in filters (case insensitive) 2001 --include stringArray Include files matching pattern 2002 --include-from stringArray Read file include patterns from file (use - to read from stdin) 2003 --max-age Duration Only transfer files younger than this in s or suffix ms|s|m|h|d|w|M|y (default off) 2004 --max-depth int If set limits the recursion depth to this (default -1) 2005 --max-size SizeSuffix Only transfer files smaller than this in KiB or suffix B|K|M|G|T|P (default off) 2006 --metadata-exclude stringArray Exclude metadatas matching pattern 2007 --metadata-exclude-from stringArray Read metadata exclude patterns from file (use - to read from stdin) 2008 --metadata-filter stringArray Add a metadata filtering rule 2009 --metadata-filter-from stringArray Read metadata filtering patterns from a file (use - to read from stdin) 2010 --metadata-include stringArray Include metadatas matching pattern 2011 --metadata-include-from stringArray Read metadata include patterns from file (use - to read from stdin) 2012 --min-age Duration Only transfer files older than this in s or suffix ms|s|m|h|d|w|M|y (default off) 2013 --min-size SizeSuffix Only transfer files bigger than this in KiB or suffix B|K|M|G|T|P (default off) 2014 \f[R] 2015 .fi 2016 .SS Listing Options 2017 .PP 2018 Flags for listing directories. 2019 .IP 2020 .nf 2021 \f[C] 2022 --default-time Time Time to show if modtime is unknown for files and directories (default 2000-01-01T00:00:00Z) 2023 --fast-list Use recursive list if available; uses more memory but fewer transactions 2024 \f[R] 2025 .fi 2026 .PP 2027 See the global flags page (https://rclone.org/flags/) for global options 2028 not listed here. 2029 .SH SEE ALSO 2030 .IP \[bu] 2 2031 rclone (https://rclone.org/commands/rclone/) - Show help for rclone 2032 commands, flags and backends. 2033 .SH rclone delete 2034 .PP 2035 Remove the files in path. 2036 .SS Synopsis 2037 .PP 2038 Remove the files in path. 2039 Unlike purge (https://rclone.org/commands/rclone_purge/) it obeys 2040 include/exclude filters so can be used to selectively delete files. 2041 .PP 2042 \f[C]rclone delete\f[R] only deletes files but leaves the directory 2043 structure alone. 2044 If you want to delete a directory and all of its contents use the 2045 purge (https://rclone.org/commands/rclone_purge/) command. 2046 .PP 2047 If you supply the \f[C]--rmdirs\f[R] flag, it will remove all empty 2048 directories along with it. 2049 You can also use the separate command 2050 rmdir (https://rclone.org/commands/rclone_rmdir/) or 2051 rmdirs (https://rclone.org/commands/rclone_rmdirs/) to delete empty 2052 directories only. 2053 .PP 2054 For example, to delete all files bigger than 100 MiB, you may first want 2055 to check what would be deleted (use either): 2056 .IP 2057 .nf 2058 \f[C] 2059 rclone --min-size 100M lsl remote:path 2060 rclone --dry-run --min-size 100M delete remote:path 2061 \f[R] 2062 .fi 2063 .PP 2064 Then proceed with the actual delete: 2065 .IP 2066 .nf 2067 \f[C] 2068 rclone --min-size 100M delete remote:path 2069 \f[R] 2070 .fi 2071 .PP 2072 That reads \[dq]delete everything with a minimum size of 100 MiB\[dq], 2073 hence delete all files bigger than 100 MiB. 2074 .PP 2075 \f[B]Important\f[R]: Since this can cause data loss, test first with the 2076 \f[C]--dry-run\f[R] or the \f[C]--interactive\f[R]/\f[C]-i\f[R] flag. 2077 .IP 2078 .nf 2079 \f[C] 2080 rclone delete remote:path [flags] 2081 \f[R] 2082 .fi 2083 .SS Options 2084 .IP 2085 .nf 2086 \f[C] 2087 -h, --help help for delete 2088 --rmdirs rmdirs removes empty directories but leaves root intact 2089 \f[R] 2090 .fi 2091 .SS Important Options 2092 .PP 2093 Important flags useful for most commands. 2094 .IP 2095 .nf 2096 \f[C] 2097 -n, --dry-run Do a trial run with no permanent changes 2098 -i, --interactive Enable interactive mode 2099 -v, --verbose count Print lots more stuff (repeat for more) 2100 \f[R] 2101 .fi 2102 .SS Filter Options 2103 .PP 2104 Flags for filtering directory listings. 2105 .IP 2106 .nf 2107 \f[C] 2108 --delete-excluded Delete files on dest excluded from sync 2109 --exclude stringArray Exclude files matching pattern 2110 --exclude-from stringArray Read file exclude patterns from file (use - to read from stdin) 2111 --exclude-if-present stringArray Exclude directories if filename is present 2112 --files-from stringArray Read list of source-file names from file (use - to read from stdin) 2113 --files-from-raw stringArray Read list of source-file names from file without any processing of lines (use - to read from stdin) 2114 -f, --filter stringArray Add a file filtering rule 2115 --filter-from stringArray Read file filtering patterns from a file (use - to read from stdin) 2116 --ignore-case Ignore case in filters (case insensitive) 2117 --include stringArray Include files matching pattern 2118 --include-from stringArray Read file include patterns from file (use - to read from stdin) 2119 --max-age Duration Only transfer files younger than this in s or suffix ms|s|m|h|d|w|M|y (default off) 2120 --max-depth int If set limits the recursion depth to this (default -1) 2121 --max-size SizeSuffix Only transfer files smaller than this in KiB or suffix B|K|M|G|T|P (default off) 2122 --metadata-exclude stringArray Exclude metadatas matching pattern 2123 --metadata-exclude-from stringArray Read metadata exclude patterns from file (use - to read from stdin) 2124 --metadata-filter stringArray Add a metadata filtering rule 2125 --metadata-filter-from stringArray Read metadata filtering patterns from a file (use - to read from stdin) 2126 --metadata-include stringArray Include metadatas matching pattern 2127 --metadata-include-from stringArray Read metadata include patterns from file (use - to read from stdin) 2128 --min-age Duration Only transfer files older than this in s or suffix ms|s|m|h|d|w|M|y (default off) 2129 --min-size SizeSuffix Only transfer files bigger than this in KiB or suffix B|K|M|G|T|P (default off) 2130 \f[R] 2131 .fi 2132 .SS Listing Options 2133 .PP 2134 Flags for listing directories. 2135 .IP 2136 .nf 2137 \f[C] 2138 --default-time Time Time to show if modtime is unknown for files and directories (default 2000-01-01T00:00:00Z) 2139 --fast-list Use recursive list if available; uses more memory but fewer transactions 2140 \f[R] 2141 .fi 2142 .PP 2143 See the global flags page (https://rclone.org/flags/) for global options 2144 not listed here. 2145 .SH SEE ALSO 2146 .IP \[bu] 2 2147 rclone (https://rclone.org/commands/rclone/) - Show help for rclone 2148 commands, flags and backends. 2149 .SH rclone purge 2150 .PP 2151 Remove the path and all of its contents. 2152 .SS Synopsis 2153 .PP 2154 Remove the path and all of its contents. 2155 Note that this does not obey include/exclude filters - everything will 2156 be removed. 2157 Use the delete (https://rclone.org/commands/rclone_delete/) command if 2158 you want to selectively delete files. 2159 To delete empty directories only, use command 2160 rmdir (https://rclone.org/commands/rclone_rmdir/) or 2161 rmdirs (https://rclone.org/commands/rclone_rmdirs/). 2162 .PP 2163 \f[B]Important\f[R]: Since this can cause data loss, test first with the 2164 \f[C]--dry-run\f[R] or the \f[C]--interactive\f[R]/\f[C]-i\f[R] flag. 2165 .IP 2166 .nf 2167 \f[C] 2168 rclone purge remote:path [flags] 2169 \f[R] 2170 .fi 2171 .SS Options 2172 .IP 2173 .nf 2174 \f[C] 2175 -h, --help help for purge 2176 \f[R] 2177 .fi 2178 .SS Important Options 2179 .PP 2180 Important flags useful for most commands. 2181 .IP 2182 .nf 2183 \f[C] 2184 -n, --dry-run Do a trial run with no permanent changes 2185 -i, --interactive Enable interactive mode 2186 -v, --verbose count Print lots more stuff (repeat for more) 2187 \f[R] 2188 .fi 2189 .PP 2190 See the global flags page (https://rclone.org/flags/) for global options 2191 not listed here. 2192 .SH SEE ALSO 2193 .IP \[bu] 2 2194 rclone (https://rclone.org/commands/rclone/) - Show help for rclone 2195 commands, flags and backends. 2196 .SH rclone mkdir 2197 .PP 2198 Make the path if it doesn\[aq]t already exist. 2199 .IP 2200 .nf 2201 \f[C] 2202 rclone mkdir remote:path [flags] 2203 \f[R] 2204 .fi 2205 .SS Options 2206 .IP 2207 .nf 2208 \f[C] 2209 -h, --help help for mkdir 2210 \f[R] 2211 .fi 2212 .SS Important Options 2213 .PP 2214 Important flags useful for most commands. 2215 .IP 2216 .nf 2217 \f[C] 2218 -n, --dry-run Do a trial run with no permanent changes 2219 -i, --interactive Enable interactive mode 2220 -v, --verbose count Print lots more stuff (repeat for more) 2221 \f[R] 2222 .fi 2223 .PP 2224 See the global flags page (https://rclone.org/flags/) for global options 2225 not listed here. 2226 .SH SEE ALSO 2227 .IP \[bu] 2 2228 rclone (https://rclone.org/commands/rclone/) - Show help for rclone 2229 commands, flags and backends. 2230 .SH rclone rmdir 2231 .PP 2232 Remove the empty directory at path. 2233 .SS Synopsis 2234 .PP 2235 This removes empty directory given by path. 2236 Will not remove the path if it has any objects in it, not even empty 2237 subdirectories. 2238 Use command rmdirs (https://rclone.org/commands/rclone_rmdirs/) (or 2239 delete (https://rclone.org/commands/rclone_delete/) with option 2240 \f[C]--rmdirs\f[R]) to do that. 2241 .PP 2242 To delete a path and any objects in it, use 2243 purge (https://rclone.org/commands/rclone_purge/) command. 2244 .IP 2245 .nf 2246 \f[C] 2247 rclone rmdir remote:path [flags] 2248 \f[R] 2249 .fi 2250 .SS Options 2251 .IP 2252 .nf 2253 \f[C] 2254 -h, --help help for rmdir 2255 \f[R] 2256 .fi 2257 .SS Important Options 2258 .PP 2259 Important flags useful for most commands. 2260 .IP 2261 .nf 2262 \f[C] 2263 -n, --dry-run Do a trial run with no permanent changes 2264 -i, --interactive Enable interactive mode 2265 -v, --verbose count Print lots more stuff (repeat for more) 2266 \f[R] 2267 .fi 2268 .PP 2269 See the global flags page (https://rclone.org/flags/) for global options 2270 not listed here. 2271 .SH SEE ALSO 2272 .IP \[bu] 2 2273 rclone (https://rclone.org/commands/rclone/) - Show help for rclone 2274 commands, flags and backends. 2275 .SH rclone check 2276 .PP 2277 Checks the files in the source and destination match. 2278 .SS Synopsis 2279 .PP 2280 Checks the files in the source and destination match. 2281 It compares sizes and hashes (MD5 or SHA1) and logs a report of files 2282 that don\[aq]t match. 2283 It doesn\[aq]t alter the source or destination. 2284 .PP 2285 For the crypt (https://rclone.org/crypt/) remote there is a dedicated 2286 command, cryptcheck (https://rclone.org/commands/rclone_cryptcheck/), 2287 that are able to check the checksums of the encrypted files. 2288 .PP 2289 If you supply the \f[C]--size-only\f[R] flag, it will only compare the 2290 sizes not the hashes as well. 2291 Use this for a quick check. 2292 .PP 2293 If you supply the \f[C]--download\f[R] flag, it will download the data 2294 from both remotes and check them against each other on the fly. 2295 This can be useful for remotes that don\[aq]t support hashes or if you 2296 really want to check all the data. 2297 .PP 2298 If you supply the \f[C]--checkfile HASH\f[R] flag with a valid hash 2299 name, the \f[C]source:path\f[R] must point to a text file in the SUM 2300 format. 2301 .PP 2302 If you supply the \f[C]--one-way\f[R] flag, it will only check that 2303 files in the source match the files in the destination, not the other 2304 way around. 2305 This means that extra files in the destination that are not in the 2306 source will not be detected. 2307 .PP 2308 The \f[C]--differ\f[R], \f[C]--missing-on-dst\f[R], 2309 \f[C]--missing-on-src\f[R], \f[C]--match\f[R] and \f[C]--error\f[R] 2310 flags write paths, one per line, to the file name (or stdout if it is 2311 \f[C]-\f[R]) supplied. 2312 What they write is described in the help below. 2313 For example \f[C]--differ\f[R] will write all paths which are present on 2314 both the source and destination but different. 2315 .PP 2316 The \f[C]--combined\f[R] flag will write a file (or stdout) which 2317 contains all file paths with a symbol and then a space and then the path 2318 to tell you what happened to it. 2319 These are reminiscent of diff files. 2320 .IP \[bu] 2 2321 \f[C]= path\f[R] means path was found in source and destination and was 2322 identical 2323 .IP \[bu] 2 2324 \[ga]- path\[ga] means path was missing on the source, so only in the 2325 destination 2326 .IP \[bu] 2 2327 \[ga]+ path\[ga] means path was missing on the destination, so only in 2328 the source 2329 .IP \[bu] 2 2330 \[ga]* path\[ga] means path was present in source and destination but 2331 different. 2332 .IP \[bu] 2 2333 \f[C]! path\f[R] means there was an error reading or hashing the source 2334 or dest. 2335 .PP 2336 The default number of parallel checks is 8. 2337 See the --checkers=N (https://rclone.org/docs/#checkers-n) option for 2338 more information. 2339 .IP 2340 .nf 2341 \f[C] 2342 rclone check source:path dest:path [flags] 2343 \f[R] 2344 .fi 2345 .SS Options 2346 .IP 2347 .nf 2348 \f[C] 2349 -C, --checkfile string Treat source:path as a SUM file with hashes of given type 2350 --combined string Make a combined report of changes to this file 2351 --differ string Report all non-matching files to this file 2352 --download Check by downloading rather than with hash 2353 --error string Report all files with errors (hashing or reading) to this file 2354 -h, --help help for check 2355 --match string Report all matching files to this file 2356 --missing-on-dst string Report all files missing from the destination to this file 2357 --missing-on-src string Report all files missing from the source to this file 2358 --one-way Check one way only, source files must exist on remote 2359 \f[R] 2360 .fi 2361 .SS Check Options 2362 .PP 2363 Flags used for \f[C]rclone check\f[R]. 2364 .IP 2365 .nf 2366 \f[C] 2367 --max-backlog int Maximum number of objects in sync or check backlog (default 10000) 2368 \f[R] 2369 .fi 2370 .SS Filter Options 2371 .PP 2372 Flags for filtering directory listings. 2373 .IP 2374 .nf 2375 \f[C] 2376 --delete-excluded Delete files on dest excluded from sync 2377 --exclude stringArray Exclude files matching pattern 2378 --exclude-from stringArray Read file exclude patterns from file (use - to read from stdin) 2379 --exclude-if-present stringArray Exclude directories if filename is present 2380 --files-from stringArray Read list of source-file names from file (use - to read from stdin) 2381 --files-from-raw stringArray Read list of source-file names from file without any processing of lines (use - to read from stdin) 2382 -f, --filter stringArray Add a file filtering rule 2383 --filter-from stringArray Read file filtering patterns from a file (use - to read from stdin) 2384 --ignore-case Ignore case in filters (case insensitive) 2385 --include stringArray Include files matching pattern 2386 --include-from stringArray Read file include patterns from file (use - to read from stdin) 2387 --max-age Duration Only transfer files younger than this in s or suffix ms|s|m|h|d|w|M|y (default off) 2388 --max-depth int If set limits the recursion depth to this (default -1) 2389 --max-size SizeSuffix Only transfer files smaller than this in KiB or suffix B|K|M|G|T|P (default off) 2390 --metadata-exclude stringArray Exclude metadatas matching pattern 2391 --metadata-exclude-from stringArray Read metadata exclude patterns from file (use - to read from stdin) 2392 --metadata-filter stringArray Add a metadata filtering rule 2393 --metadata-filter-from stringArray Read metadata filtering patterns from a file (use - to read from stdin) 2394 --metadata-include stringArray Include metadatas matching pattern 2395 --metadata-include-from stringArray Read metadata include patterns from file (use - to read from stdin) 2396 --min-age Duration Only transfer files older than this in s or suffix ms|s|m|h|d|w|M|y (default off) 2397 --min-size SizeSuffix Only transfer files bigger than this in KiB or suffix B|K|M|G|T|P (default off) 2398 \f[R] 2399 .fi 2400 .SS Listing Options 2401 .PP 2402 Flags for listing directories. 2403 .IP 2404 .nf 2405 \f[C] 2406 --default-time Time Time to show if modtime is unknown for files and directories (default 2000-01-01T00:00:00Z) 2407 --fast-list Use recursive list if available; uses more memory but fewer transactions 2408 \f[R] 2409 .fi 2410 .PP 2411 See the global flags page (https://rclone.org/flags/) for global options 2412 not listed here. 2413 .SH SEE ALSO 2414 .IP \[bu] 2 2415 rclone (https://rclone.org/commands/rclone/) - Show help for rclone 2416 commands, flags and backends. 2417 .SH rclone ls 2418 .PP 2419 List the objects in the path with size and path. 2420 .SS Synopsis 2421 .PP 2422 Lists the objects in the source path to standard output in a human 2423 readable format with size and path. 2424 Recurses by default. 2425 .PP 2426 Eg 2427 .IP 2428 .nf 2429 \f[C] 2430 $ rclone ls swift:bucket 2431 60295 bevajer5jef 2432 90613 canole 2433 94467 diwogej7 2434 37600 fubuwic 2435 \f[R] 2436 .fi 2437 .PP 2438 Any of the filtering options can be applied to this command. 2439 .PP 2440 There are several related list commands 2441 .IP \[bu] 2 2442 \f[C]ls\f[R] to list size and path of objects only 2443 .IP \[bu] 2 2444 \f[C]lsl\f[R] to list modification time, size and path of objects only 2445 .IP \[bu] 2 2446 \f[C]lsd\f[R] to list directories only 2447 .IP \[bu] 2 2448 \f[C]lsf\f[R] to list objects and directories in easy to parse format 2449 .IP \[bu] 2 2450 \f[C]lsjson\f[R] to list objects and directories in JSON format 2451 .PP 2452 \f[C]ls\f[R],\f[C]lsl\f[R],\f[C]lsd\f[R] are designed to be 2453 human-readable. 2454 \f[C]lsf\f[R] is designed to be human and machine-readable. 2455 \f[C]lsjson\f[R] is designed to be machine-readable. 2456 .PP 2457 Note that \f[C]ls\f[R] and \f[C]lsl\f[R] recurse by default - use 2458 \f[C]--max-depth 1\f[R] to stop the recursion. 2459 .PP 2460 The other list commands \f[C]lsd\f[R],\f[C]lsf\f[R],\f[C]lsjson\f[R] do 2461 not recurse by default - use \f[C]-R\f[R] to make them recurse. 2462 .PP 2463 Listing a nonexistent directory will produce an error except for remotes 2464 which can\[aq]t have empty directories (e.g. 2465 s3, swift, or gcs - the bucket-based remotes). 2466 .IP 2467 .nf 2468 \f[C] 2469 rclone ls remote:path [flags] 2470 \f[R] 2471 .fi 2472 .SS Options 2473 .IP 2474 .nf 2475 \f[C] 2476 -h, --help help for ls 2477 \f[R] 2478 .fi 2479 .SS Filter Options 2480 .PP 2481 Flags for filtering directory listings. 2482 .IP 2483 .nf 2484 \f[C] 2485 --delete-excluded Delete files on dest excluded from sync 2486 --exclude stringArray Exclude files matching pattern 2487 --exclude-from stringArray Read file exclude patterns from file (use - to read from stdin) 2488 --exclude-if-present stringArray Exclude directories if filename is present 2489 --files-from stringArray Read list of source-file names from file (use - to read from stdin) 2490 --files-from-raw stringArray Read list of source-file names from file without any processing of lines (use - to read from stdin) 2491 -f, --filter stringArray Add a file filtering rule 2492 --filter-from stringArray Read file filtering patterns from a file (use - to read from stdin) 2493 --ignore-case Ignore case in filters (case insensitive) 2494 --include stringArray Include files matching pattern 2495 --include-from stringArray Read file include patterns from file (use - to read from stdin) 2496 --max-age Duration Only transfer files younger than this in s or suffix ms|s|m|h|d|w|M|y (default off) 2497 --max-depth int If set limits the recursion depth to this (default -1) 2498 --max-size SizeSuffix Only transfer files smaller than this in KiB or suffix B|K|M|G|T|P (default off) 2499 --metadata-exclude stringArray Exclude metadatas matching pattern 2500 --metadata-exclude-from stringArray Read metadata exclude patterns from file (use - to read from stdin) 2501 --metadata-filter stringArray Add a metadata filtering rule 2502 --metadata-filter-from stringArray Read metadata filtering patterns from a file (use - to read from stdin) 2503 --metadata-include stringArray Include metadatas matching pattern 2504 --metadata-include-from stringArray Read metadata include patterns from file (use - to read from stdin) 2505 --min-age Duration Only transfer files older than this in s or suffix ms|s|m|h|d|w|M|y (default off) 2506 --min-size SizeSuffix Only transfer files bigger than this in KiB or suffix B|K|M|G|T|P (default off) 2507 \f[R] 2508 .fi 2509 .SS Listing Options 2510 .PP 2511 Flags for listing directories. 2512 .IP 2513 .nf 2514 \f[C] 2515 --default-time Time Time to show if modtime is unknown for files and directories (default 2000-01-01T00:00:00Z) 2516 --fast-list Use recursive list if available; uses more memory but fewer transactions 2517 \f[R] 2518 .fi 2519 .PP 2520 See the global flags page (https://rclone.org/flags/) for global options 2521 not listed here. 2522 .SH SEE ALSO 2523 .IP \[bu] 2 2524 rclone (https://rclone.org/commands/rclone/) - Show help for rclone 2525 commands, flags and backends. 2526 .SH rclone lsd 2527 .PP 2528 List all directories/containers/buckets in the path. 2529 .SS Synopsis 2530 .PP 2531 Lists the directories in the source path to standard output. 2532 Does not recurse by default. 2533 Use the \f[C]-R\f[R] flag to recurse. 2534 .PP 2535 This command lists the total size of the directory (if known, -1 if 2536 not), the modification time (if known, the current time if not), the 2537 number of objects in the directory (if known, -1 if not) and the name of 2538 the directory, Eg 2539 .IP 2540 .nf 2541 \f[C] 2542 $ rclone lsd swift: 2543 494000 2018-04-26 08:43:20 10000 10000files 2544 65 2018-04-26 08:43:20 1 1File 2545 \f[R] 2546 .fi 2547 .PP 2548 Or 2549 .IP 2550 .nf 2551 \f[C] 2552 $ rclone lsd drive:test 2553 -1 2016-10-17 17:41:53 -1 1000files 2554 -1 2017-01-03 14:40:54 -1 2500files 2555 -1 2017-07-08 14:39:28 -1 4000files 2556 \f[R] 2557 .fi 2558 .PP 2559 If you just want the directory names use 2560 \f[C]rclone lsf --dirs-only\f[R]. 2561 .PP 2562 Any of the filtering options can be applied to this command. 2563 .PP 2564 There are several related list commands 2565 .IP \[bu] 2 2566 \f[C]ls\f[R] to list size and path of objects only 2567 .IP \[bu] 2 2568 \f[C]lsl\f[R] to list modification time, size and path of objects only 2569 .IP \[bu] 2 2570 \f[C]lsd\f[R] to list directories only 2571 .IP \[bu] 2 2572 \f[C]lsf\f[R] to list objects and directories in easy to parse format 2573 .IP \[bu] 2 2574 \f[C]lsjson\f[R] to list objects and directories in JSON format 2575 .PP 2576 \f[C]ls\f[R],\f[C]lsl\f[R],\f[C]lsd\f[R] are designed to be 2577 human-readable. 2578 \f[C]lsf\f[R] is designed to be human and machine-readable. 2579 \f[C]lsjson\f[R] is designed to be machine-readable. 2580 .PP 2581 Note that \f[C]ls\f[R] and \f[C]lsl\f[R] recurse by default - use 2582 \f[C]--max-depth 1\f[R] to stop the recursion. 2583 .PP 2584 The other list commands \f[C]lsd\f[R],\f[C]lsf\f[R],\f[C]lsjson\f[R] do 2585 not recurse by default - use \f[C]-R\f[R] to make them recurse. 2586 .PP 2587 Listing a nonexistent directory will produce an error except for remotes 2588 which can\[aq]t have empty directories (e.g. 2589 s3, swift, or gcs - the bucket-based remotes). 2590 .IP 2591 .nf 2592 \f[C] 2593 rclone lsd remote:path [flags] 2594 \f[R] 2595 .fi 2596 .SS Options 2597 .IP 2598 .nf 2599 \f[C] 2600 -h, --help help for lsd 2601 -R, --recursive Recurse into the listing 2602 \f[R] 2603 .fi 2604 .SS Filter Options 2605 .PP 2606 Flags for filtering directory listings. 2607 .IP 2608 .nf 2609 \f[C] 2610 --delete-excluded Delete files on dest excluded from sync 2611 --exclude stringArray Exclude files matching pattern 2612 --exclude-from stringArray Read file exclude patterns from file (use - to read from stdin) 2613 --exclude-if-present stringArray Exclude directories if filename is present 2614 --files-from stringArray Read list of source-file names from file (use - to read from stdin) 2615 --files-from-raw stringArray Read list of source-file names from file without any processing of lines (use - to read from stdin) 2616 -f, --filter stringArray Add a file filtering rule 2617 --filter-from stringArray Read file filtering patterns from a file (use - to read from stdin) 2618 --ignore-case Ignore case in filters (case insensitive) 2619 --include stringArray Include files matching pattern 2620 --include-from stringArray Read file include patterns from file (use - to read from stdin) 2621 --max-age Duration Only transfer files younger than this in s or suffix ms|s|m|h|d|w|M|y (default off) 2622 --max-depth int If set limits the recursion depth to this (default -1) 2623 --max-size SizeSuffix Only transfer files smaller than this in KiB or suffix B|K|M|G|T|P (default off) 2624 --metadata-exclude stringArray Exclude metadatas matching pattern 2625 --metadata-exclude-from stringArray Read metadata exclude patterns from file (use - to read from stdin) 2626 --metadata-filter stringArray Add a metadata filtering rule 2627 --metadata-filter-from stringArray Read metadata filtering patterns from a file (use - to read from stdin) 2628 --metadata-include stringArray Include metadatas matching pattern 2629 --metadata-include-from stringArray Read metadata include patterns from file (use - to read from stdin) 2630 --min-age Duration Only transfer files older than this in s or suffix ms|s|m|h|d|w|M|y (default off) 2631 --min-size SizeSuffix Only transfer files bigger than this in KiB or suffix B|K|M|G|T|P (default off) 2632 \f[R] 2633 .fi 2634 .SS Listing Options 2635 .PP 2636 Flags for listing directories. 2637 .IP 2638 .nf 2639 \f[C] 2640 --default-time Time Time to show if modtime is unknown for files and directories (default 2000-01-01T00:00:00Z) 2641 --fast-list Use recursive list if available; uses more memory but fewer transactions 2642 \f[R] 2643 .fi 2644 .PP 2645 See the global flags page (https://rclone.org/flags/) for global options 2646 not listed here. 2647 .SH SEE ALSO 2648 .IP \[bu] 2 2649 rclone (https://rclone.org/commands/rclone/) - Show help for rclone 2650 commands, flags and backends. 2651 .SH rclone lsl 2652 .PP 2653 List the objects in path with modification time, size and path. 2654 .SS Synopsis 2655 .PP 2656 Lists the objects in the source path to standard output in a human 2657 readable format with modification time, size and path. 2658 Recurses by default. 2659 .PP 2660 Eg 2661 .IP 2662 .nf 2663 \f[C] 2664 $ rclone lsl swift:bucket 2665 60295 2016-06-25 18:55:41.062626927 bevajer5jef 2666 90613 2016-06-25 18:55:43.302607074 canole 2667 94467 2016-06-25 18:55:43.046609333 diwogej7 2668 37600 2016-06-25 18:55:40.814629136 fubuwic 2669 \f[R] 2670 .fi 2671 .PP 2672 Any of the filtering options can be applied to this command. 2673 .PP 2674 There are several related list commands 2675 .IP \[bu] 2 2676 \f[C]ls\f[R] to list size and path of objects only 2677 .IP \[bu] 2 2678 \f[C]lsl\f[R] to list modification time, size and path of objects only 2679 .IP \[bu] 2 2680 \f[C]lsd\f[R] to list directories only 2681 .IP \[bu] 2 2682 \f[C]lsf\f[R] to list objects and directories in easy to parse format 2683 .IP \[bu] 2 2684 \f[C]lsjson\f[R] to list objects and directories in JSON format 2685 .PP 2686 \f[C]ls\f[R],\f[C]lsl\f[R],\f[C]lsd\f[R] are designed to be 2687 human-readable. 2688 \f[C]lsf\f[R] is designed to be human and machine-readable. 2689 \f[C]lsjson\f[R] is designed to be machine-readable. 2690 .PP 2691 Note that \f[C]ls\f[R] and \f[C]lsl\f[R] recurse by default - use 2692 \f[C]--max-depth 1\f[R] to stop the recursion. 2693 .PP 2694 The other list commands \f[C]lsd\f[R],\f[C]lsf\f[R],\f[C]lsjson\f[R] do 2695 not recurse by default - use \f[C]-R\f[R] to make them recurse. 2696 .PP 2697 Listing a nonexistent directory will produce an error except for remotes 2698 which can\[aq]t have empty directories (e.g. 2699 s3, swift, or gcs - the bucket-based remotes). 2700 .IP 2701 .nf 2702 \f[C] 2703 rclone lsl remote:path [flags] 2704 \f[R] 2705 .fi 2706 .SS Options 2707 .IP 2708 .nf 2709 \f[C] 2710 -h, --help help for lsl 2711 \f[R] 2712 .fi 2713 .SS Filter Options 2714 .PP 2715 Flags for filtering directory listings. 2716 .IP 2717 .nf 2718 \f[C] 2719 --delete-excluded Delete files on dest excluded from sync 2720 --exclude stringArray Exclude files matching pattern 2721 --exclude-from stringArray Read file exclude patterns from file (use - to read from stdin) 2722 --exclude-if-present stringArray Exclude directories if filename is present 2723 --files-from stringArray Read list of source-file names from file (use - to read from stdin) 2724 --files-from-raw stringArray Read list of source-file names from file without any processing of lines (use - to read from stdin) 2725 -f, --filter stringArray Add a file filtering rule 2726 --filter-from stringArray Read file filtering patterns from a file (use - to read from stdin) 2727 --ignore-case Ignore case in filters (case insensitive) 2728 --include stringArray Include files matching pattern 2729 --include-from stringArray Read file include patterns from file (use - to read from stdin) 2730 --max-age Duration Only transfer files younger than this in s or suffix ms|s|m|h|d|w|M|y (default off) 2731 --max-depth int If set limits the recursion depth to this (default -1) 2732 --max-size SizeSuffix Only transfer files smaller than this in KiB or suffix B|K|M|G|T|P (default off) 2733 --metadata-exclude stringArray Exclude metadatas matching pattern 2734 --metadata-exclude-from stringArray Read metadata exclude patterns from file (use - to read from stdin) 2735 --metadata-filter stringArray Add a metadata filtering rule 2736 --metadata-filter-from stringArray Read metadata filtering patterns from a file (use - to read from stdin) 2737 --metadata-include stringArray Include metadatas matching pattern 2738 --metadata-include-from stringArray Read metadata include patterns from file (use - to read from stdin) 2739 --min-age Duration Only transfer files older than this in s or suffix ms|s|m|h|d|w|M|y (default off) 2740 --min-size SizeSuffix Only transfer files bigger than this in KiB or suffix B|K|M|G|T|P (default off) 2741 \f[R] 2742 .fi 2743 .SS Listing Options 2744 .PP 2745 Flags for listing directories. 2746 .IP 2747 .nf 2748 \f[C] 2749 --default-time Time Time to show if modtime is unknown for files and directories (default 2000-01-01T00:00:00Z) 2750 --fast-list Use recursive list if available; uses more memory but fewer transactions 2751 \f[R] 2752 .fi 2753 .PP 2754 See the global flags page (https://rclone.org/flags/) for global options 2755 not listed here. 2756 .SH SEE ALSO 2757 .IP \[bu] 2 2758 rclone (https://rclone.org/commands/rclone/) - Show help for rclone 2759 commands, flags and backends. 2760 .SH rclone md5sum 2761 .PP 2762 Produces an md5sum file for all the objects in the path. 2763 .SS Synopsis 2764 .PP 2765 Produces an md5sum file for all the objects in the path. 2766 This is in the same format as the standard md5sum tool produces. 2767 .PP 2768 By default, the hash is requested from the remote. 2769 If MD5 is not supported by the remote, no hash will be returned. 2770 With the download flag, the file will be downloaded from the remote and 2771 hashed locally enabling MD5 for any remote. 2772 .PP 2773 For other algorithms, see the 2774 hashsum (https://rclone.org/commands/rclone_hashsum/) command. 2775 Running \f[C]rclone md5sum remote:path\f[R] is equivalent to running 2776 \f[C]rclone hashsum MD5 remote:path\f[R]. 2777 .PP 2778 This command can also hash data received on standard input (stdin), by 2779 not passing a remote:path, or by passing a hyphen as remote:path when 2780 there is data to read (if not, the hyphen will be treated literally, as 2781 a relative path). 2782 .IP 2783 .nf 2784 \f[C] 2785 rclone md5sum remote:path [flags] 2786 \f[R] 2787 .fi 2788 .SS Options 2789 .IP 2790 .nf 2791 \f[C] 2792 --base64 Output base64 encoded hashsum 2793 -C, --checkfile string Validate hashes against a given SUM file instead of printing them 2794 --download Download the file and hash it locally; if this flag is not specified, the hash is requested from the remote 2795 -h, --help help for md5sum 2796 --output-file string Output hashsums to a file rather than the terminal 2797 \f[R] 2798 .fi 2799 .SS Filter Options 2800 .PP 2801 Flags for filtering directory listings. 2802 .IP 2803 .nf 2804 \f[C] 2805 --delete-excluded Delete files on dest excluded from sync 2806 --exclude stringArray Exclude files matching pattern 2807 --exclude-from stringArray Read file exclude patterns from file (use - to read from stdin) 2808 --exclude-if-present stringArray Exclude directories if filename is present 2809 --files-from stringArray Read list of source-file names from file (use - to read from stdin) 2810 --files-from-raw stringArray Read list of source-file names from file without any processing of lines (use - to read from stdin) 2811 -f, --filter stringArray Add a file filtering rule 2812 --filter-from stringArray Read file filtering patterns from a file (use - to read from stdin) 2813 --ignore-case Ignore case in filters (case insensitive) 2814 --include stringArray Include files matching pattern 2815 --include-from stringArray Read file include patterns from file (use - to read from stdin) 2816 --max-age Duration Only transfer files younger than this in s or suffix ms|s|m|h|d|w|M|y (default off) 2817 --max-depth int If set limits the recursion depth to this (default -1) 2818 --max-size SizeSuffix Only transfer files smaller than this in KiB or suffix B|K|M|G|T|P (default off) 2819 --metadata-exclude stringArray Exclude metadatas matching pattern 2820 --metadata-exclude-from stringArray Read metadata exclude patterns from file (use - to read from stdin) 2821 --metadata-filter stringArray Add a metadata filtering rule 2822 --metadata-filter-from stringArray Read metadata filtering patterns from a file (use - to read from stdin) 2823 --metadata-include stringArray Include metadatas matching pattern 2824 --metadata-include-from stringArray Read metadata include patterns from file (use - to read from stdin) 2825 --min-age Duration Only transfer files older than this in s or suffix ms|s|m|h|d|w|M|y (default off) 2826 --min-size SizeSuffix Only transfer files bigger than this in KiB or suffix B|K|M|G|T|P (default off) 2827 \f[R] 2828 .fi 2829 .SS Listing Options 2830 .PP 2831 Flags for listing directories. 2832 .IP 2833 .nf 2834 \f[C] 2835 --default-time Time Time to show if modtime is unknown for files and directories (default 2000-01-01T00:00:00Z) 2836 --fast-list Use recursive list if available; uses more memory but fewer transactions 2837 \f[R] 2838 .fi 2839 .PP 2840 See the global flags page (https://rclone.org/flags/) for global options 2841 not listed here. 2842 .SH SEE ALSO 2843 .IP \[bu] 2 2844 rclone (https://rclone.org/commands/rclone/) - Show help for rclone 2845 commands, flags and backends. 2846 .SH rclone sha1sum 2847 .PP 2848 Produces an sha1sum file for all the objects in the path. 2849 .SS Synopsis 2850 .PP 2851 Produces an sha1sum file for all the objects in the path. 2852 This is in the same format as the standard sha1sum tool produces. 2853 .PP 2854 By default, the hash is requested from the remote. 2855 If SHA-1 is not supported by the remote, no hash will be returned. 2856 With the download flag, the file will be downloaded from the remote and 2857 hashed locally enabling SHA-1 for any remote. 2858 .PP 2859 For other algorithms, see the 2860 hashsum (https://rclone.org/commands/rclone_hashsum/) command. 2861 Running \f[C]rclone sha1sum remote:path\f[R] is equivalent to running 2862 \f[C]rclone hashsum SHA1 remote:path\f[R]. 2863 .PP 2864 This command can also hash data received on standard input (stdin), by 2865 not passing a remote:path, or by passing a hyphen as remote:path when 2866 there is data to read (if not, the hyphen will be treated literally, as 2867 a relative path). 2868 .PP 2869 This command can also hash data received on STDIN, if not passing a 2870 remote:path. 2871 .IP 2872 .nf 2873 \f[C] 2874 rclone sha1sum remote:path [flags] 2875 \f[R] 2876 .fi 2877 .SS Options 2878 .IP 2879 .nf 2880 \f[C] 2881 --base64 Output base64 encoded hashsum 2882 -C, --checkfile string Validate hashes against a given SUM file instead of printing them 2883 --download Download the file and hash it locally; if this flag is not specified, the hash is requested from the remote 2884 -h, --help help for sha1sum 2885 --output-file string Output hashsums to a file rather than the terminal 2886 \f[R] 2887 .fi 2888 .SS Filter Options 2889 .PP 2890 Flags for filtering directory listings. 2891 .IP 2892 .nf 2893 \f[C] 2894 --delete-excluded Delete files on dest excluded from sync 2895 --exclude stringArray Exclude files matching pattern 2896 --exclude-from stringArray Read file exclude patterns from file (use - to read from stdin) 2897 --exclude-if-present stringArray Exclude directories if filename is present 2898 --files-from stringArray Read list of source-file names from file (use - to read from stdin) 2899 --files-from-raw stringArray Read list of source-file names from file without any processing of lines (use - to read from stdin) 2900 -f, --filter stringArray Add a file filtering rule 2901 --filter-from stringArray Read file filtering patterns from a file (use - to read from stdin) 2902 --ignore-case Ignore case in filters (case insensitive) 2903 --include stringArray Include files matching pattern 2904 --include-from stringArray Read file include patterns from file (use - to read from stdin) 2905 --max-age Duration Only transfer files younger than this in s or suffix ms|s|m|h|d|w|M|y (default off) 2906 --max-depth int If set limits the recursion depth to this (default -1) 2907 --max-size SizeSuffix Only transfer files smaller than this in KiB or suffix B|K|M|G|T|P (default off) 2908 --metadata-exclude stringArray Exclude metadatas matching pattern 2909 --metadata-exclude-from stringArray Read metadata exclude patterns from file (use - to read from stdin) 2910 --metadata-filter stringArray Add a metadata filtering rule 2911 --metadata-filter-from stringArray Read metadata filtering patterns from a file (use - to read from stdin) 2912 --metadata-include stringArray Include metadatas matching pattern 2913 --metadata-include-from stringArray Read metadata include patterns from file (use - to read from stdin) 2914 --min-age Duration Only transfer files older than this in s or suffix ms|s|m|h|d|w|M|y (default off) 2915 --min-size SizeSuffix Only transfer files bigger than this in KiB or suffix B|K|M|G|T|P (default off) 2916 \f[R] 2917 .fi 2918 .SS Listing Options 2919 .PP 2920 Flags for listing directories. 2921 .IP 2922 .nf 2923 \f[C] 2924 --default-time Time Time to show if modtime is unknown for files and directories (default 2000-01-01T00:00:00Z) 2925 --fast-list Use recursive list if available; uses more memory but fewer transactions 2926 \f[R] 2927 .fi 2928 .PP 2929 See the global flags page (https://rclone.org/flags/) for global options 2930 not listed here. 2931 .SH SEE ALSO 2932 .IP \[bu] 2 2933 rclone (https://rclone.org/commands/rclone/) - Show help for rclone 2934 commands, flags and backends. 2935 .SH rclone size 2936 .PP 2937 Prints the total size and number of objects in remote:path. 2938 .SS Synopsis 2939 .PP 2940 Counts objects in the path and calculates the total size. 2941 Prints the result to standard output. 2942 .PP 2943 By default the output is in human-readable format, but shows values in 2944 both human-readable format as well as the raw numbers (global option 2945 \f[C]--human-readable\f[R] is not considered). 2946 Use option \f[C]--json\f[R] to format output as JSON instead. 2947 .PP 2948 Recurses by default, use \f[C]--max-depth 1\f[R] to stop the recursion. 2949 .PP 2950 Some backends do not always provide file sizes, see for example Google 2951 Photos (https://rclone.org/googlephotos/#size) and Google 2952 Docs (https://rclone.org/drive/#limitations-of-google-docs). 2953 Rclone will then show a notice in the log indicating how many such files 2954 were encountered, and count them in as empty files in the output of the 2955 size command. 2956 .IP 2957 .nf 2958 \f[C] 2959 rclone size remote:path [flags] 2960 \f[R] 2961 .fi 2962 .SS Options 2963 .IP 2964 .nf 2965 \f[C] 2966 -h, --help help for size 2967 --json Format output as JSON 2968 \f[R] 2969 .fi 2970 .SS Filter Options 2971 .PP 2972 Flags for filtering directory listings. 2973 .IP 2974 .nf 2975 \f[C] 2976 --delete-excluded Delete files on dest excluded from sync 2977 --exclude stringArray Exclude files matching pattern 2978 --exclude-from stringArray Read file exclude patterns from file (use - to read from stdin) 2979 --exclude-if-present stringArray Exclude directories if filename is present 2980 --files-from stringArray Read list of source-file names from file (use - to read from stdin) 2981 --files-from-raw stringArray Read list of source-file names from file without any processing of lines (use - to read from stdin) 2982 -f, --filter stringArray Add a file filtering rule 2983 --filter-from stringArray Read file filtering patterns from a file (use - to read from stdin) 2984 --ignore-case Ignore case in filters (case insensitive) 2985 --include stringArray Include files matching pattern 2986 --include-from stringArray Read file include patterns from file (use - to read from stdin) 2987 --max-age Duration Only transfer files younger than this in s or suffix ms|s|m|h|d|w|M|y (default off) 2988 --max-depth int If set limits the recursion depth to this (default -1) 2989 --max-size SizeSuffix Only transfer files smaller than this in KiB or suffix B|K|M|G|T|P (default off) 2990 --metadata-exclude stringArray Exclude metadatas matching pattern 2991 --metadata-exclude-from stringArray Read metadata exclude patterns from file (use - to read from stdin) 2992 --metadata-filter stringArray Add a metadata filtering rule 2993 --metadata-filter-from stringArray Read metadata filtering patterns from a file (use - to read from stdin) 2994 --metadata-include stringArray Include metadatas matching pattern 2995 --metadata-include-from stringArray Read metadata include patterns from file (use - to read from stdin) 2996 --min-age Duration Only transfer files older than this in s or suffix ms|s|m|h|d|w|M|y (default off) 2997 --min-size SizeSuffix Only transfer files bigger than this in KiB or suffix B|K|M|G|T|P (default off) 2998 \f[R] 2999 .fi 3000 .SS Listing Options 3001 .PP 3002 Flags for listing directories. 3003 .IP 3004 .nf 3005 \f[C] 3006 --default-time Time Time to show if modtime is unknown for files and directories (default 2000-01-01T00:00:00Z) 3007 --fast-list Use recursive list if available; uses more memory but fewer transactions 3008 \f[R] 3009 .fi 3010 .PP 3011 See the global flags page (https://rclone.org/flags/) for global options 3012 not listed here. 3013 .SH SEE ALSO 3014 .IP \[bu] 2 3015 rclone (https://rclone.org/commands/rclone/) - Show help for rclone 3016 commands, flags and backends. 3017 .SH rclone version 3018 .PP 3019 Show the version number. 3020 .SS Synopsis 3021 .PP 3022 Show the rclone version number, the go version, the build target OS and 3023 architecture, the runtime OS and kernel version and bitness, build tags 3024 and the type of executable (static or dynamic). 3025 .PP 3026 For example: 3027 .IP 3028 .nf 3029 \f[C] 3030 $ rclone version 3031 rclone v1.55.0 3032 - os/version: ubuntu 18.04 (64 bit) 3033 - os/kernel: 4.15.0-136-generic (x86_64) 3034 - os/type: linux 3035 - os/arch: amd64 3036 - go/version: go1.16 3037 - go/linking: static 3038 - go/tags: none 3039 \f[R] 3040 .fi 3041 .PP 3042 Note: before rclone version 1.55 the os/type and os/arch lines were 3043 merged, and the \[dq]go/version\[dq] line was tagged as \[dq]go 3044 version\[dq]. 3045 .PP 3046 If you supply the --check flag, then it will do an online check to 3047 compare your version with the latest release and the latest beta. 3048 .IP 3049 .nf 3050 \f[C] 3051 $ rclone version --check 3052 yours: 1.42.0.6 3053 latest: 1.42 (released 2018-06-16) 3054 beta: 1.42.0.5 (released 2018-06-17) 3055 \f[R] 3056 .fi 3057 .PP 3058 Or 3059 .IP 3060 .nf 3061 \f[C] 3062 $ rclone version --check 3063 yours: 1.41 3064 latest: 1.42 (released 2018-06-16) 3065 upgrade: https://downloads.rclone.org/v1.42 3066 beta: 1.42.0.5 (released 2018-06-17) 3067 upgrade: https://beta.rclone.org/v1.42-005-g56e1e820 3068 \f[R] 3069 .fi 3070 .IP 3071 .nf 3072 \f[C] 3073 rclone version [flags] 3074 \f[R] 3075 .fi 3076 .SS Options 3077 .IP 3078 .nf 3079 \f[C] 3080 --check Check for new version 3081 -h, --help help for version 3082 \f[R] 3083 .fi 3084 .PP 3085 See the global flags page (https://rclone.org/flags/) for global options 3086 not listed here. 3087 .SH SEE ALSO 3088 .IP \[bu] 2 3089 rclone (https://rclone.org/commands/rclone/) - Show help for rclone 3090 commands, flags and backends. 3091 .SH rclone cleanup 3092 .PP 3093 Clean up the remote if possible. 3094 .SS Synopsis 3095 .PP 3096 Clean up the remote if possible. 3097 Empty the trash or delete old file versions. 3098 Not supported by all remotes. 3099 .IP 3100 .nf 3101 \f[C] 3102 rclone cleanup remote:path [flags] 3103 \f[R] 3104 .fi 3105 .SS Options 3106 .IP 3107 .nf 3108 \f[C] 3109 -h, --help help for cleanup 3110 \f[R] 3111 .fi 3112 .SS Important Options 3113 .PP 3114 Important flags useful for most commands. 3115 .IP 3116 .nf 3117 \f[C] 3118 -n, --dry-run Do a trial run with no permanent changes 3119 -i, --interactive Enable interactive mode 3120 -v, --verbose count Print lots more stuff (repeat for more) 3121 \f[R] 3122 .fi 3123 .PP 3124 See the global flags page (https://rclone.org/flags/) for global options 3125 not listed here. 3126 .SH SEE ALSO 3127 .IP \[bu] 2 3128 rclone (https://rclone.org/commands/rclone/) - Show help for rclone 3129 commands, flags and backends. 3130 .SH rclone dedupe 3131 .PP 3132 Interactively find duplicate filenames and delete/rename them. 3133 .SS Synopsis 3134 .PP 3135 By default \f[C]dedupe\f[R] interactively finds files with duplicate 3136 names and offers to delete all but one or rename them to be different. 3137 This is known as deduping by name. 3138 .PP 3139 Deduping by name is only useful with a small group of backends (e.g. 3140 Google Drive, Opendrive) that can have duplicate file names. 3141 It can be run on wrapping backends (e.g. 3142 crypt) if they wrap a backend which supports duplicate file names. 3143 .PP 3144 However if \f[C]--by-hash\f[R] is passed in then dedupe will find files 3145 with duplicate hashes instead which will work on any backend which 3146 supports at least one hash. 3147 This can be used to find files with duplicate content. 3148 This is known as deduping by hash. 3149 .PP 3150 If deduping by name, first rclone will merge directories with the same 3151 name. 3152 It will do this iteratively until all the identically named directories 3153 have been merged. 3154 .PP 3155 Next, if deduping by name, for every group of duplicate file names / 3156 hashes, it will delete all but one identical file it finds without 3157 confirmation. 3158 This means that for most duplicated files the \f[C]dedupe\f[R] command 3159 will not be interactive. 3160 .PP 3161 \f[C]dedupe\f[R] considers files to be identical if they have the same 3162 file path and the same hash. 3163 If the backend does not support hashes (e.g. 3164 crypt wrapping Google Drive) then they will never be found to be 3165 identical. 3166 If you use the \f[C]--size-only\f[R] flag then files will be considered 3167 identical if they have the same size (any hash will be ignored). 3168 This can be useful on crypt backends which do not support hashes. 3169 .PP 3170 Next rclone will resolve the remaining duplicates. 3171 Exactly which action is taken depends on the dedupe mode. 3172 By default, rclone will interactively query the user for each one. 3173 .PP 3174 \f[B]Important\f[R]: Since this can cause data loss, test first with the 3175 \f[C]--dry-run\f[R] or the \f[C]--interactive\f[R]/\f[C]-i\f[R] flag. 3176 .PP 3177 Here is an example run. 3178 .PP 3179 Before - with duplicates 3180 .IP 3181 .nf 3182 \f[C] 3183 $ rclone lsl drive:dupes 3184 6048320 2016-03-05 16:23:16.798000000 one.txt 3185 6048320 2016-03-05 16:23:11.775000000 one.txt 3186 564374 2016-03-05 16:23:06.731000000 one.txt 3187 6048320 2016-03-05 16:18:26.092000000 one.txt 3188 6048320 2016-03-05 16:22:46.185000000 two.txt 3189 1744073 2016-03-05 16:22:38.104000000 two.txt 3190 564374 2016-03-05 16:22:52.118000000 two.txt 3191 \f[R] 3192 .fi 3193 .PP 3194 Now the \f[C]dedupe\f[R] session 3195 .IP 3196 .nf 3197 \f[C] 3198 $ rclone dedupe drive:dupes 3199 2016/03/05 16:24:37 Google drive root \[aq]dupes\[aq]: Looking for duplicates using interactive mode. 3200 one.txt: Found 4 files with duplicate names 3201 one.txt: Deleting 2/3 identical duplicates (MD5 \[dq]1eedaa9fe86fd4b8632e2ac549403b36\[dq]) 3202 one.txt: 2 duplicates remain 3203 1: 6048320 bytes, 2016-03-05 16:23:16.798000000, MD5 1eedaa9fe86fd4b8632e2ac549403b36 3204 2: 564374 bytes, 2016-03-05 16:23:06.731000000, MD5 7594e7dc9fc28f727c42ee3e0749de81 3205 s) Skip and do nothing 3206 k) Keep just one (choose which in next step) 3207 r) Rename all to be different (by changing file.jpg to file-1.jpg) 3208 s/k/r> k 3209 Enter the number of the file to keep> 1 3210 one.txt: Deleted 1 extra copies 3211 two.txt: Found 3 files with duplicate names 3212 two.txt: 3 duplicates remain 3213 1: 564374 bytes, 2016-03-05 16:22:52.118000000, MD5 7594e7dc9fc28f727c42ee3e0749de81 3214 2: 6048320 bytes, 2016-03-05 16:22:46.185000000, MD5 1eedaa9fe86fd4b8632e2ac549403b36 3215 3: 1744073 bytes, 2016-03-05 16:22:38.104000000, MD5 851957f7fb6f0bc4ce76be966d336802 3216 s) Skip and do nothing 3217 k) Keep just one (choose which in next step) 3218 r) Rename all to be different (by changing file.jpg to file-1.jpg) 3219 s/k/r> r 3220 two-1.txt: renamed from: two.txt 3221 two-2.txt: renamed from: two.txt 3222 two-3.txt: renamed from: two.txt 3223 \f[R] 3224 .fi 3225 .PP 3226 The result being 3227 .IP 3228 .nf 3229 \f[C] 3230 $ rclone lsl drive:dupes 3231 6048320 2016-03-05 16:23:16.798000000 one.txt 3232 564374 2016-03-05 16:22:52.118000000 two-1.txt 3233 6048320 2016-03-05 16:22:46.185000000 two-2.txt 3234 1744073 2016-03-05 16:22:38.104000000 two-3.txt 3235 \f[R] 3236 .fi 3237 .PP 3238 Dedupe can be run non interactively using the \f[C]--dedupe-mode\f[R] 3239 flag or by using an extra parameter with the same value 3240 .IP \[bu] 2 3241 \f[C]--dedupe-mode interactive\f[R] - interactive as above. 3242 .IP \[bu] 2 3243 \f[C]--dedupe-mode skip\f[R] - removes identical files then skips 3244 anything left. 3245 .IP \[bu] 2 3246 \f[C]--dedupe-mode first\f[R] - removes identical files then keeps the 3247 first one. 3248 .IP \[bu] 2 3249 \f[C]--dedupe-mode newest\f[R] - removes identical files then keeps the 3250 newest one. 3251 .IP \[bu] 2 3252 \f[C]--dedupe-mode oldest\f[R] - removes identical files then keeps the 3253 oldest one. 3254 .IP \[bu] 2 3255 \f[C]--dedupe-mode largest\f[R] - removes identical files then keeps the 3256 largest one. 3257 .IP \[bu] 2 3258 \f[C]--dedupe-mode smallest\f[R] - removes identical files then keeps 3259 the smallest one. 3260 .IP \[bu] 2 3261 \f[C]--dedupe-mode rename\f[R] - removes identical files then renames 3262 the rest to be different. 3263 .IP \[bu] 2 3264 \f[C]--dedupe-mode list\f[R] - lists duplicate dirs and files only and 3265 changes nothing. 3266 .PP 3267 For example, to rename all the identically named photos in your Google 3268 Photos directory, do 3269 .IP 3270 .nf 3271 \f[C] 3272 rclone dedupe --dedupe-mode rename \[dq]drive:Google Photos\[dq] 3273 \f[R] 3274 .fi 3275 .PP 3276 Or 3277 .IP 3278 .nf 3279 \f[C] 3280 rclone dedupe rename \[dq]drive:Google Photos\[dq] 3281 \f[R] 3282 .fi 3283 .IP 3284 .nf 3285 \f[C] 3286 rclone dedupe [mode] remote:path [flags] 3287 \f[R] 3288 .fi 3289 .SS Options 3290 .IP 3291 .nf 3292 \f[C] 3293 --by-hash Find identical hashes rather than names 3294 --dedupe-mode string Dedupe mode interactive|skip|first|newest|oldest|largest|smallest|rename (default \[dq]interactive\[dq]) 3295 -h, --help help for dedupe 3296 \f[R] 3297 .fi 3298 .SS Important Options 3299 .PP 3300 Important flags useful for most commands. 3301 .IP 3302 .nf 3303 \f[C] 3304 -n, --dry-run Do a trial run with no permanent changes 3305 -i, --interactive Enable interactive mode 3306 -v, --verbose count Print lots more stuff (repeat for more) 3307 \f[R] 3308 .fi 3309 .PP 3310 See the global flags page (https://rclone.org/flags/) for global options 3311 not listed here. 3312 .SH SEE ALSO 3313 .IP \[bu] 2 3314 rclone (https://rclone.org/commands/rclone/) - Show help for rclone 3315 commands, flags and backends. 3316 .SH rclone about 3317 .PP 3318 Get quota information from the remote. 3319 .SS Synopsis 3320 .PP 3321 \f[C]rclone about\f[R] prints quota information about a remote to 3322 standard output. 3323 The output is typically used, free, quota and trash contents. 3324 .PP 3325 E.g. 3326 Typical output from \f[C]rclone about remote:\f[R] is: 3327 .IP 3328 .nf 3329 \f[C] 3330 Total: 17 GiB 3331 Used: 7.444 GiB 3332 Free: 1.315 GiB 3333 Trashed: 100.000 MiB 3334 Other: 8.241 GiB 3335 \f[R] 3336 .fi 3337 .PP 3338 Where the fields are: 3339 .IP \[bu] 2 3340 Total: Total size available. 3341 .IP \[bu] 2 3342 Used: Total size used. 3343 .IP \[bu] 2 3344 Free: Total space available to this user. 3345 .IP \[bu] 2 3346 Trashed: Total space used by trash. 3347 .IP \[bu] 2 3348 Other: Total amount in other storage (e.g. 3349 Gmail, Google Photos). 3350 .IP \[bu] 2 3351 Objects: Total number of objects in the storage. 3352 .PP 3353 All sizes are in number of bytes. 3354 .PP 3355 Applying a \f[C]--full\f[R] flag to the command prints the bytes in 3356 full, e.g. 3357 .IP 3358 .nf 3359 \f[C] 3360 Total: 18253611008 3361 Used: 7993453766 3362 Free: 1411001220 3363 Trashed: 104857602 3364 Other: 8849156022 3365 \f[R] 3366 .fi 3367 .PP 3368 A \f[C]--json\f[R] flag generates conveniently machine-readable output, 3369 e.g. 3370 .IP 3371 .nf 3372 \f[C] 3373 { 3374 \[dq]total\[dq]: 18253611008, 3375 \[dq]used\[dq]: 7993453766, 3376 \[dq]trashed\[dq]: 104857602, 3377 \[dq]other\[dq]: 8849156022, 3378 \[dq]free\[dq]: 1411001220 3379 } 3380 \f[R] 3381 .fi 3382 .PP 3383 Not all backends print all fields. 3384 Information is not included if it is not provided by a backend. 3385 Where the value is unlimited it is omitted. 3386 .PP 3387 Some backends does not support the \f[C]rclone about\f[R] command at 3388 all, see complete list in 3389 documentation (https://rclone.org/overview/#optional-features). 3390 .IP 3391 .nf 3392 \f[C] 3393 rclone about remote: [flags] 3394 \f[R] 3395 .fi 3396 .SS Options 3397 .IP 3398 .nf 3399 \f[C] 3400 --full Full numbers instead of human-readable 3401 -h, --help help for about 3402 --json Format output as JSON 3403 \f[R] 3404 .fi 3405 .PP 3406 See the global flags page (https://rclone.org/flags/) for global options 3407 not listed here. 3408 .SH SEE ALSO 3409 .IP \[bu] 2 3410 rclone (https://rclone.org/commands/rclone/) - Show help for rclone 3411 commands, flags and backends. 3412 .SH rclone authorize 3413 .PP 3414 Remote authorization. 3415 .SS Synopsis 3416 .PP 3417 Remote authorization. 3418 Used to authorize a remote or headless rclone from a machine with a 3419 browser - use as instructed by rclone config. 3420 .PP 3421 Use --auth-no-open-browser to prevent rclone to open auth link in 3422 default browser automatically. 3423 .PP 3424 Use --template to generate HTML output via a custom Go template. 3425 If a blank string is provided as an argument to this flag, the default 3426 template is used. 3427 .IP 3428 .nf 3429 \f[C] 3430 rclone authorize [flags] 3431 \f[R] 3432 .fi 3433 .SS Options 3434 .IP 3435 .nf 3436 \f[C] 3437 --auth-no-open-browser Do not automatically open auth link in default browser 3438 -h, --help help for authorize 3439 --template string The path to a custom Go template for generating HTML responses 3440 \f[R] 3441 .fi 3442 .PP 3443 See the global flags page (https://rclone.org/flags/) for global options 3444 not listed here. 3445 .SH SEE ALSO 3446 .IP \[bu] 2 3447 rclone (https://rclone.org/commands/rclone/) - Show help for rclone 3448 commands, flags and backends. 3449 .SH rclone backend 3450 .PP 3451 Run a backend-specific command. 3452 .SS Synopsis 3453 .PP 3454 This runs a backend-specific command. 3455 The commands themselves (except for \[dq]help\[dq] and 3456 \[dq]features\[dq]) are defined by the backends and you should see the 3457 backend docs for definitions. 3458 .PP 3459 You can discover what commands a backend implements by using 3460 .IP 3461 .nf 3462 \f[C] 3463 rclone backend help remote: 3464 rclone backend help <backendname> 3465 \f[R] 3466 .fi 3467 .PP 3468 You can also discover information about the backend using (see 3469 operations/fsinfo (https://rclone.org/rc/#operations-fsinfo) in the 3470 remote control docs for more info). 3471 .IP 3472 .nf 3473 \f[C] 3474 rclone backend features remote: 3475 \f[R] 3476 .fi 3477 .PP 3478 Pass options to the backend command with -o. 3479 This should be key=value or key, e.g.: 3480 .IP 3481 .nf 3482 \f[C] 3483 rclone backend stats remote:path stats -o format=json -o long 3484 \f[R] 3485 .fi 3486 .PP 3487 Pass arguments to the backend by placing them on the end of the line 3488 .IP 3489 .nf 3490 \f[C] 3491 rclone backend cleanup remote:path file1 file2 file3 3492 \f[R] 3493 .fi 3494 .PP 3495 Note to run these commands on a running backend then see 3496 backend/command (https://rclone.org/rc/#backend-command) in the rc docs. 3497 .IP 3498 .nf 3499 \f[C] 3500 rclone backend <command> remote:path [opts] <args> [flags] 3501 \f[R] 3502 .fi 3503 .SS Options 3504 .IP 3505 .nf 3506 \f[C] 3507 -h, --help help for backend 3508 --json Always output in JSON format 3509 -o, --option stringArray Option in the form name=value or name 3510 \f[R] 3511 .fi 3512 .SS Important Options 3513 .PP 3514 Important flags useful for most commands. 3515 .IP 3516 .nf 3517 \f[C] 3518 -n, --dry-run Do a trial run with no permanent changes 3519 -i, --interactive Enable interactive mode 3520 -v, --verbose count Print lots more stuff (repeat for more) 3521 \f[R] 3522 .fi 3523 .PP 3524 See the global flags page (https://rclone.org/flags/) for global options 3525 not listed here. 3526 .SH SEE ALSO 3527 .IP \[bu] 2 3528 rclone (https://rclone.org/commands/rclone/) - Show help for rclone 3529 commands, flags and backends. 3530 .SH rclone bisync 3531 .PP 3532 Perform bidirectional synchronization between two paths. 3533 .SS Synopsis 3534 .PP 3535 Perform bidirectional synchronization between two paths. 3536 .PP 3537 Bisync (https://rclone.org/bisync/) provides a bidirectional cloud sync 3538 solution in rclone. 3539 It retains the Path1 and Path2 filesystem listings from the prior run. 3540 On each successive run it will: - list files on Path1 and Path2, and 3541 check for changes on each side. 3542 Changes include \f[C]New\f[R], \f[C]Newer\f[R], \f[C]Older\f[R], and 3543 \f[C]Deleted\f[R] files. 3544 - Propagate changes on Path1 to Path2, and vice-versa. 3545 .PP 3546 Bisync is \f[B]in beta\f[R] and is considered an \f[B]advanced 3547 command\f[R], so use with care. 3548 Make sure you have read and understood the entire 3549 manual (https://rclone.org/bisync) (especially the 3550 Limitations (https://rclone.org/bisync/#limitations) section) before 3551 using, or data loss can result. 3552 Questions can be asked in the Rclone Forum (https://forum.rclone.org/). 3553 .PP 3554 See full bisync description (https://rclone.org/bisync/) for details. 3555 .IP 3556 .nf 3557 \f[C] 3558 rclone bisync remote1:path1 remote2:path2 [flags] 3559 \f[R] 3560 .fi 3561 .SS Options 3562 .IP 3563 .nf 3564 \f[C] 3565 --backup-dir1 string --backup-dir for Path1. Must be a non-overlapping path on the same remote. 3566 --backup-dir2 string --backup-dir for Path2. Must be a non-overlapping path on the same remote. 3567 --check-access Ensure expected RCLONE_TEST files are found on both Path1 and Path2 filesystems, else abort. 3568 --check-filename string Filename for --check-access (default: RCLONE_TEST) 3569 --check-sync string Controls comparison of final listings: true|false|only (default: true) (default \[dq]true\[dq]) 3570 --compare string Comma-separated list of bisync-specific compare options ex. \[aq]size,modtime,checksum\[aq] (default: \[aq]size,modtime\[aq]) 3571 --conflict-loser ConflictLoserAction Action to take on the loser of a sync conflict (when there is a winner) or on both files (when there is no winner): , num, pathname, delete (default: num) 3572 --conflict-resolve string Automatically resolve conflicts by preferring the version that is: none, path1, path2, newer, older, larger, smaller (default: none) (default \[dq]none\[dq]) 3573 --conflict-suffix string Suffix to use when renaming a --conflict-loser. Can be either one string or two comma-separated strings to assign different suffixes to Path1/Path2. (default: \[aq]conflict\[aq]) 3574 --create-empty-src-dirs Sync creation and deletion of empty directories. (Not compatible with --remove-empty-dirs) 3575 --download-hash Compute hash by downloading when otherwise unavailable. (warning: may be slow and use lots of data!) 3576 --filters-file string Read filtering patterns from a file 3577 --force Bypass --max-delete safety check and run the sync. Consider using with --verbose 3578 -h, --help help for bisync 3579 --ignore-listing-checksum Do not use checksums for listings (add --ignore-checksum to additionally skip post-copy checksum checks) 3580 --max-lock Duration Consider lock files older than this to be expired (default: 0 (never expire)) (minimum: 2m) (default 0s) 3581 --no-cleanup Retain working files (useful for troubleshooting and testing). 3582 --no-slow-hash Ignore listing checksums only on backends where they are slow 3583 --recover Automatically recover from interruptions without requiring --resync. 3584 --remove-empty-dirs Remove ALL empty directories at the final cleanup step. 3585 --resilient Allow future runs to retry after certain less-serious errors, instead of requiring --resync. Use at your own risk! 3586 -1, --resync Performs the resync run. Equivalent to --resync-mode path1. Consider using --verbose or --dry-run first. 3587 --resync-mode string During resync, prefer the version that is: path1, path2, newer, older, larger, smaller (default: path1 if --resync, otherwise none for no resync.) (default \[dq]none\[dq]) 3588 --slow-hash-sync-only Ignore slow checksums for listings and deltas, but still consider them during sync calls. 3589 --workdir string Use custom working dir - useful for testing. (default: {WORKDIR}) 3590 \f[R] 3591 .fi 3592 .SS Copy Options 3593 .PP 3594 Flags for anything which can Copy a file. 3595 .IP 3596 .nf 3597 \f[C] 3598 --check-first Do all the checks before starting transfers 3599 -c, --checksum Check for changes with size & checksum (if available, or fallback to size only). 3600 --compare-dest stringArray Include additional comma separated server-side paths during comparison 3601 --copy-dest stringArray Implies --compare-dest but also copies files from paths into destination 3602 --cutoff-mode HARD|SOFT|CAUTIOUS Mode to stop transfers when reaching the max transfer limit HARD|SOFT|CAUTIOUS (default HARD) 3603 --ignore-case-sync Ignore case when synchronizing 3604 --ignore-checksum Skip post copy check of checksums 3605 --ignore-existing Skip all files that exist on destination 3606 --ignore-size Ignore size when skipping use modtime or checksum 3607 -I, --ignore-times Don\[aq]t skip items that match size and time - transfer all unconditionally 3608 --immutable Do not modify files, fail if existing files have been modified 3609 --inplace Download directly to destination file instead of atomic download to temp/rename 3610 --max-backlog int Maximum number of objects in sync or check backlog (default 10000) 3611 --max-duration Duration Maximum duration rclone will transfer data for (default 0s) 3612 --max-transfer SizeSuffix Maximum size of data to transfer (default off) 3613 -M, --metadata If set, preserve metadata when copying objects 3614 --modify-window Duration Max time diff to be considered the same (default 1ns) 3615 --multi-thread-chunk-size SizeSuffix Chunk size for multi-thread downloads / uploads, if not set by filesystem (default 64Mi) 3616 --multi-thread-cutoff SizeSuffix Use multi-thread downloads for files above this size (default 256Mi) 3617 --multi-thread-streams int Number of streams to use for multi-thread downloads (default 4) 3618 --multi-thread-write-buffer-size SizeSuffix In memory buffer size for writing when in multi-thread mode (default 128Ki) 3619 --no-check-dest Don\[aq]t check the destination, copy regardless 3620 --no-traverse Don\[aq]t traverse destination file system on copy 3621 --no-update-dir-modtime Don\[aq]t update directory modification times 3622 --no-update-modtime Don\[aq]t update destination modtime if files identical 3623 --order-by string Instructions on how to order the transfers, e.g. \[aq]size,descending\[aq] 3624 --partial-suffix string Add partial-suffix to temporary file name when --inplace is not used (default \[dq].partial\[dq]) 3625 --refresh-times Refresh the modtime of remote files 3626 --server-side-across-configs Allow server-side operations (e.g. copy) to work across different configs 3627 --size-only Skip based on size only, not modtime or checksum 3628 --streaming-upload-cutoff SizeSuffix Cutoff for switching to chunked upload if file size is unknown, upload starts after reaching cutoff or when file ends (default 100Ki) 3629 -u, --update Skip files that are newer on the destination 3630 \f[R] 3631 .fi 3632 .SS Important Options 3633 .PP 3634 Important flags useful for most commands. 3635 .IP 3636 .nf 3637 \f[C] 3638 -n, --dry-run Do a trial run with no permanent changes 3639 -i, --interactive Enable interactive mode 3640 -v, --verbose count Print lots more stuff (repeat for more) 3641 \f[R] 3642 .fi 3643 .SS Filter Options 3644 .PP 3645 Flags for filtering directory listings. 3646 .IP 3647 .nf 3648 \f[C] 3649 --delete-excluded Delete files on dest excluded from sync 3650 --exclude stringArray Exclude files matching pattern 3651 --exclude-from stringArray Read file exclude patterns from file (use - to read from stdin) 3652 --exclude-if-present stringArray Exclude directories if filename is present 3653 --files-from stringArray Read list of source-file names from file (use - to read from stdin) 3654 --files-from-raw stringArray Read list of source-file names from file without any processing of lines (use - to read from stdin) 3655 -f, --filter stringArray Add a file filtering rule 3656 --filter-from stringArray Read file filtering patterns from a file (use - to read from stdin) 3657 --ignore-case Ignore case in filters (case insensitive) 3658 --include stringArray Include files matching pattern 3659 --include-from stringArray Read file include patterns from file (use - to read from stdin) 3660 --max-age Duration Only transfer files younger than this in s or suffix ms|s|m|h|d|w|M|y (default off) 3661 --max-depth int If set limits the recursion depth to this (default -1) 3662 --max-size SizeSuffix Only transfer files smaller than this in KiB or suffix B|K|M|G|T|P (default off) 3663 --metadata-exclude stringArray Exclude metadatas matching pattern 3664 --metadata-exclude-from stringArray Read metadata exclude patterns from file (use - to read from stdin) 3665 --metadata-filter stringArray Add a metadata filtering rule 3666 --metadata-filter-from stringArray Read metadata filtering patterns from a file (use - to read from stdin) 3667 --metadata-include stringArray Include metadatas matching pattern 3668 --metadata-include-from stringArray Read metadata include patterns from file (use - to read from stdin) 3669 --min-age Duration Only transfer files older than this in s or suffix ms|s|m|h|d|w|M|y (default off) 3670 --min-size SizeSuffix Only transfer files bigger than this in KiB or suffix B|K|M|G|T|P (default off) 3671 \f[R] 3672 .fi 3673 .PP 3674 See the global flags page (https://rclone.org/flags/) for global options 3675 not listed here. 3676 .SH SEE ALSO 3677 .IP \[bu] 2 3678 rclone (https://rclone.org/commands/rclone/) - Show help for rclone 3679 commands, flags and backends. 3680 .SH rclone cat 3681 .PP 3682 Concatenates any files and sends them to stdout. 3683 .SS Synopsis 3684 .PP 3685 rclone cat sends any files to standard output. 3686 .PP 3687 You can use it like this to output a single file 3688 .IP 3689 .nf 3690 \f[C] 3691 rclone cat remote:path/to/file 3692 \f[R] 3693 .fi 3694 .PP 3695 Or like this to output any file in dir or its subdirectories. 3696 .IP 3697 .nf 3698 \f[C] 3699 rclone cat remote:path/to/dir 3700 \f[R] 3701 .fi 3702 .PP 3703 Or like this to output any .txt files in dir or its subdirectories. 3704 .IP 3705 .nf 3706 \f[C] 3707 rclone --include \[dq]*.txt\[dq] cat remote:path/to/dir 3708 \f[R] 3709 .fi 3710 .PP 3711 Use the \f[C]--head\f[R] flag to print characters only at the start, 3712 \f[C]--tail\f[R] for the end and \f[C]--offset\f[R] and 3713 \f[C]--count\f[R] to print a section in the middle. 3714 Note that if offset is negative it will count from the end, so 3715 \f[C]--offset -1 --count 1\f[R] is equivalent to \f[C]--tail 1\f[R]. 3716 .PP 3717 Use the \f[C]--separator\f[R] flag to print a separator value between 3718 files. 3719 Be sure to shell-escape special characters. 3720 For example, to print a newline between files, use: 3721 .IP \[bu] 2 3722 bash: 3723 .RS 2 3724 .IP 3725 .nf 3726 \f[C] 3727 rclone --include \[dq]*.txt\[dq] --separator $\[aq]\[rs]n\[aq] cat remote:path/to/dir 3728 \f[R] 3729 .fi 3730 .RE 3731 .IP \[bu] 2 3732 powershell: 3733 .RS 2 3734 .IP 3735 .nf 3736 \f[C] 3737 rclone --include \[dq]*.txt\[dq] --separator \[dq]\[ga]n\[dq] cat remote:path/to/dir 3738 \f[R] 3739 .fi 3740 .RE 3741 .IP 3742 .nf 3743 \f[C] 3744 rclone cat remote:path [flags] 3745 \f[R] 3746 .fi 3747 .SS Options 3748 .IP 3749 .nf 3750 \f[C] 3751 --count int Only print N characters (default -1) 3752 --discard Discard the output instead of printing 3753 --head int Only print the first N characters 3754 -h, --help help for cat 3755 --offset int Start printing at offset N (or from end if -ve) 3756 --separator string Separator to use between objects when printing multiple files 3757 --tail int Only print the last N characters 3758 \f[R] 3759 .fi 3760 .SS Filter Options 3761 .PP 3762 Flags for filtering directory listings. 3763 .IP 3764 .nf 3765 \f[C] 3766 --delete-excluded Delete files on dest excluded from sync 3767 --exclude stringArray Exclude files matching pattern 3768 --exclude-from stringArray Read file exclude patterns from file (use - to read from stdin) 3769 --exclude-if-present stringArray Exclude directories if filename is present 3770 --files-from stringArray Read list of source-file names from file (use - to read from stdin) 3771 --files-from-raw stringArray Read list of source-file names from file without any processing of lines (use - to read from stdin) 3772 -f, --filter stringArray Add a file filtering rule 3773 --filter-from stringArray Read file filtering patterns from a file (use - to read from stdin) 3774 --ignore-case Ignore case in filters (case insensitive) 3775 --include stringArray Include files matching pattern 3776 --include-from stringArray Read file include patterns from file (use - to read from stdin) 3777 --max-age Duration Only transfer files younger than this in s or suffix ms|s|m|h|d|w|M|y (default off) 3778 --max-depth int If set limits the recursion depth to this (default -1) 3779 --max-size SizeSuffix Only transfer files smaller than this in KiB or suffix B|K|M|G|T|P (default off) 3780 --metadata-exclude stringArray Exclude metadatas matching pattern 3781 --metadata-exclude-from stringArray Read metadata exclude patterns from file (use - to read from stdin) 3782 --metadata-filter stringArray Add a metadata filtering rule 3783 --metadata-filter-from stringArray Read metadata filtering patterns from a file (use - to read from stdin) 3784 --metadata-include stringArray Include metadatas matching pattern 3785 --metadata-include-from stringArray Read metadata include patterns from file (use - to read from stdin) 3786 --min-age Duration Only transfer files older than this in s or suffix ms|s|m|h|d|w|M|y (default off) 3787 --min-size SizeSuffix Only transfer files bigger than this in KiB or suffix B|K|M|G|T|P (default off) 3788 \f[R] 3789 .fi 3790 .SS Listing Options 3791 .PP 3792 Flags for listing directories. 3793 .IP 3794 .nf 3795 \f[C] 3796 --default-time Time Time to show if modtime is unknown for files and directories (default 2000-01-01T00:00:00Z) 3797 --fast-list Use recursive list if available; uses more memory but fewer transactions 3798 \f[R] 3799 .fi 3800 .PP 3801 See the global flags page (https://rclone.org/flags/) for global options 3802 not listed here. 3803 .SH SEE ALSO 3804 .IP \[bu] 2 3805 rclone (https://rclone.org/commands/rclone/) - Show help for rclone 3806 commands, flags and backends. 3807 .SH rclone checksum 3808 .PP 3809 Checks the files in the destination against a SUM file. 3810 .SS Synopsis 3811 .PP 3812 Checks that hashsums of destination files match the SUM file. 3813 It compares hashes (MD5, SHA1, etc) and logs a report of files which 3814 don\[aq]t match. 3815 It doesn\[aq]t alter the file system. 3816 .PP 3817 The sumfile is treated as the source and the dst:path is treated as the 3818 destination for the purposes of the output. 3819 .PP 3820 If you supply the \f[C]--download\f[R] flag, it will download the data 3821 from the remote and calculate the content hash on the fly. 3822 This can be useful for remotes that don\[aq]t support hashes or if you 3823 really want to check all the data. 3824 .PP 3825 Note that hash values in the SUM file are treated as case insensitive. 3826 .PP 3827 If you supply the \f[C]--one-way\f[R] flag, it will only check that 3828 files in the source match the files in the destination, not the other 3829 way around. 3830 This means that extra files in the destination that are not in the 3831 source will not be detected. 3832 .PP 3833 The \f[C]--differ\f[R], \f[C]--missing-on-dst\f[R], 3834 \f[C]--missing-on-src\f[R], \f[C]--match\f[R] and \f[C]--error\f[R] 3835 flags write paths, one per line, to the file name (or stdout if it is 3836 \f[C]-\f[R]) supplied. 3837 What they write is described in the help below. 3838 For example \f[C]--differ\f[R] will write all paths which are present on 3839 both the source and destination but different. 3840 .PP 3841 The \f[C]--combined\f[R] flag will write a file (or stdout) which 3842 contains all file paths with a symbol and then a space and then the path 3843 to tell you what happened to it. 3844 These are reminiscent of diff files. 3845 .IP \[bu] 2 3846 \f[C]= path\f[R] means path was found in source and destination and was 3847 identical 3848 .IP \[bu] 2 3849 \[ga]- path\[ga] means path was missing on the source, so only in the 3850 destination 3851 .IP \[bu] 2 3852 \[ga]+ path\[ga] means path was missing on the destination, so only in 3853 the source 3854 .IP \[bu] 2 3855 \[ga]* path\[ga] means path was present in source and destination but 3856 different. 3857 .IP \[bu] 2 3858 \f[C]! path\f[R] means there was an error reading or hashing the source 3859 or dest. 3860 .PP 3861 The default number of parallel checks is 8. 3862 See the --checkers=N (https://rclone.org/docs/#checkers-n) option for 3863 more information. 3864 .IP 3865 .nf 3866 \f[C] 3867 rclone checksum <hash> sumfile dst:path [flags] 3868 \f[R] 3869 .fi 3870 .SS Options 3871 .IP 3872 .nf 3873 \f[C] 3874 --combined string Make a combined report of changes to this file 3875 --differ string Report all non-matching files to this file 3876 --download Check by hashing the contents 3877 --error string Report all files with errors (hashing or reading) to this file 3878 -h, --help help for checksum 3879 --match string Report all matching files to this file 3880 --missing-on-dst string Report all files missing from the destination to this file 3881 --missing-on-src string Report all files missing from the source to this file 3882 --one-way Check one way only, source files must exist on remote 3883 \f[R] 3884 .fi 3885 .SS Filter Options 3886 .PP 3887 Flags for filtering directory listings. 3888 .IP 3889 .nf 3890 \f[C] 3891 --delete-excluded Delete files on dest excluded from sync 3892 --exclude stringArray Exclude files matching pattern 3893 --exclude-from stringArray Read file exclude patterns from file (use - to read from stdin) 3894 --exclude-if-present stringArray Exclude directories if filename is present 3895 --files-from stringArray Read list of source-file names from file (use - to read from stdin) 3896 --files-from-raw stringArray Read list of source-file names from file without any processing of lines (use - to read from stdin) 3897 -f, --filter stringArray Add a file filtering rule 3898 --filter-from stringArray Read file filtering patterns from a file (use - to read from stdin) 3899 --ignore-case Ignore case in filters (case insensitive) 3900 --include stringArray Include files matching pattern 3901 --include-from stringArray Read file include patterns from file (use - to read from stdin) 3902 --max-age Duration Only transfer files younger than this in s or suffix ms|s|m|h|d|w|M|y (default off) 3903 --max-depth int If set limits the recursion depth to this (default -1) 3904 --max-size SizeSuffix Only transfer files smaller than this in KiB or suffix B|K|M|G|T|P (default off) 3905 --metadata-exclude stringArray Exclude metadatas matching pattern 3906 --metadata-exclude-from stringArray Read metadata exclude patterns from file (use - to read from stdin) 3907 --metadata-filter stringArray Add a metadata filtering rule 3908 --metadata-filter-from stringArray Read metadata filtering patterns from a file (use - to read from stdin) 3909 --metadata-include stringArray Include metadatas matching pattern 3910 --metadata-include-from stringArray Read metadata include patterns from file (use - to read from stdin) 3911 --min-age Duration Only transfer files older than this in s or suffix ms|s|m|h|d|w|M|y (default off) 3912 --min-size SizeSuffix Only transfer files bigger than this in KiB or suffix B|K|M|G|T|P (default off) 3913 \f[R] 3914 .fi 3915 .SS Listing Options 3916 .PP 3917 Flags for listing directories. 3918 .IP 3919 .nf 3920 \f[C] 3921 --default-time Time Time to show if modtime is unknown for files and directories (default 2000-01-01T00:00:00Z) 3922 --fast-list Use recursive list if available; uses more memory but fewer transactions 3923 \f[R] 3924 .fi 3925 .PP 3926 See the global flags page (https://rclone.org/flags/) for global options 3927 not listed here. 3928 .SH SEE ALSO 3929 .IP \[bu] 2 3930 rclone (https://rclone.org/commands/rclone/) - Show help for rclone 3931 commands, flags and backends. 3932 .SH rclone completion 3933 .PP 3934 Output completion script for a given shell. 3935 .SS Synopsis 3936 .PP 3937 Generates a shell completion script for rclone. 3938 Run with \f[C]--help\f[R] to list the supported shells. 3939 .SS Options 3940 .IP 3941 .nf 3942 \f[C] 3943 -h, --help help for completion 3944 \f[R] 3945 .fi 3946 .PP 3947 See the global flags page (https://rclone.org/flags/) for global options 3948 not listed here. 3949 .SH SEE ALSO 3950 .IP \[bu] 2 3951 rclone (https://rclone.org/commands/rclone/) - Show help for rclone 3952 commands, flags and backends. 3953 .IP \[bu] 2 3954 rclone completion 3955 bash (https://rclone.org/commands/rclone_completion_bash/) - Output bash 3956 completion script for rclone. 3957 .IP \[bu] 2 3958 rclone completion 3959 fish (https://rclone.org/commands/rclone_completion_fish/) - Output fish 3960 completion script for rclone. 3961 .IP \[bu] 2 3962 rclone completion 3963 powershell (https://rclone.org/commands/rclone_completion_powershell/) - 3964 Output powershell completion script for rclone. 3965 .IP \[bu] 2 3966 rclone completion 3967 zsh (https://rclone.org/commands/rclone_completion_zsh/) - Output zsh 3968 completion script for rclone. 3969 .SH rclone completion bash 3970 .PP 3971 Output bash completion script for rclone. 3972 .SS Synopsis 3973 .PP 3974 Generates a bash shell autocompletion script for rclone. 3975 .PP 3976 This writes to /etc/bash_completion.d/rclone by default so will probably 3977 need to be run with sudo or as root, e.g. 3978 .IP 3979 .nf 3980 \f[C] 3981 sudo rclone genautocomplete bash 3982 \f[R] 3983 .fi 3984 .PP 3985 Logout and login again to use the autocompletion scripts, or source them 3986 directly 3987 .IP 3988 .nf 3989 \f[C] 3990 \&. /etc/bash_completion 3991 \f[R] 3992 .fi 3993 .PP 3994 If you supply a command line argument the script will be written there. 3995 .PP 3996 If output_file is \[dq]-\[dq], then the output will be written to 3997 stdout. 3998 .IP 3999 .nf 4000 \f[C] 4001 rclone completion bash [output_file] [flags] 4002 \f[R] 4003 .fi 4004 .SS Options 4005 .IP 4006 .nf 4007 \f[C] 4008 -h, --help help for bash 4009 \f[R] 4010 .fi 4011 .PP 4012 See the global flags page (https://rclone.org/flags/) for global options 4013 not listed here. 4014 .SH SEE ALSO 4015 .IP \[bu] 2 4016 rclone completion (https://rclone.org/commands/rclone_completion/) - 4017 Output completion script for a given shell. 4018 .SH rclone completion fish 4019 .PP 4020 Output fish completion script for rclone. 4021 .SS Synopsis 4022 .PP 4023 Generates a fish autocompletion script for rclone. 4024 .PP 4025 This writes to /etc/fish/completions/rclone.fish by default so will 4026 probably need to be run with sudo or as root, e.g. 4027 .IP 4028 .nf 4029 \f[C] 4030 sudo rclone genautocomplete fish 4031 \f[R] 4032 .fi 4033 .PP 4034 Logout and login again to use the autocompletion scripts, or source them 4035 directly 4036 .IP 4037 .nf 4038 \f[C] 4039 \&. /etc/fish/completions/rclone.fish 4040 \f[R] 4041 .fi 4042 .PP 4043 If you supply a command line argument the script will be written there. 4044 .PP 4045 If output_file is \[dq]-\[dq], then the output will be written to 4046 stdout. 4047 .IP 4048 .nf 4049 \f[C] 4050 rclone completion fish [output_file] [flags] 4051 \f[R] 4052 .fi 4053 .SS Options 4054 .IP 4055 .nf 4056 \f[C] 4057 -h, --help help for fish 4058 \f[R] 4059 .fi 4060 .PP 4061 See the global flags page (https://rclone.org/flags/) for global options 4062 not listed here. 4063 .SH SEE ALSO 4064 .IP \[bu] 2 4065 rclone completion (https://rclone.org/commands/rclone_completion/) - 4066 Output completion script for a given shell. 4067 .SH rclone completion powershell 4068 .PP 4069 Output powershell completion script for rclone. 4070 .SS Synopsis 4071 .PP 4072 Generate the autocompletion script for powershell. 4073 .PP 4074 To load completions in your current shell session: 4075 .IP 4076 .nf 4077 \f[C] 4078 rclone completion powershell | Out-String | Invoke-Expression 4079 \f[R] 4080 .fi 4081 .PP 4082 To load completions for every new session, add the output of the above 4083 command to your powershell profile. 4084 .PP 4085 If output_file is \[dq]-\[dq] or missing, then the output will be 4086 written to stdout. 4087 .IP 4088 .nf 4089 \f[C] 4090 rclone completion powershell [output_file] [flags] 4091 \f[R] 4092 .fi 4093 .SS Options 4094 .IP 4095 .nf 4096 \f[C] 4097 -h, --help help for powershell 4098 \f[R] 4099 .fi 4100 .PP 4101 See the global flags page (https://rclone.org/flags/) for global options 4102 not listed here. 4103 .SH SEE ALSO 4104 .IP \[bu] 2 4105 rclone completion (https://rclone.org/commands/rclone_completion/) - 4106 Output completion script for a given shell. 4107 .SH rclone completion zsh 4108 .PP 4109 Output zsh completion script for rclone. 4110 .SS Synopsis 4111 .PP 4112 Generates a zsh autocompletion script for rclone. 4113 .PP 4114 This writes to /usr/share/zsh/vendor-completions/_rclone by default so 4115 will probably need to be run with sudo or as root, e.g. 4116 .IP 4117 .nf 4118 \f[C] 4119 sudo rclone genautocomplete zsh 4120 \f[R] 4121 .fi 4122 .PP 4123 Logout and login again to use the autocompletion scripts, or source them 4124 directly 4125 .IP 4126 .nf 4127 \f[C] 4128 autoload -U compinit && compinit 4129 \f[R] 4130 .fi 4131 .PP 4132 If you supply a command line argument the script will be written there. 4133 .PP 4134 If output_file is \[dq]-\[dq], then the output will be written to 4135 stdout. 4136 .IP 4137 .nf 4138 \f[C] 4139 rclone completion zsh [output_file] [flags] 4140 \f[R] 4141 .fi 4142 .SS Options 4143 .IP 4144 .nf 4145 \f[C] 4146 -h, --help help for zsh 4147 \f[R] 4148 .fi 4149 .PP 4150 See the global flags page (https://rclone.org/flags/) for global options 4151 not listed here. 4152 .SH SEE ALSO 4153 .IP \[bu] 2 4154 rclone completion (https://rclone.org/commands/rclone_completion/) - 4155 Output completion script for a given shell. 4156 .SH rclone config create 4157 .PP 4158 Create a new remote with name, type and options. 4159 .SS Synopsis 4160 .PP 4161 Create a new remote of \f[C]name\f[R] with \f[C]type\f[R] and options. 4162 The options should be passed in pairs of \f[C]key\f[R] \f[C]value\f[R] 4163 or as \f[C]key=value\f[R]. 4164 .PP 4165 For example, to make a swift remote of name myremote using auto config 4166 you would do: 4167 .IP 4168 .nf 4169 \f[C] 4170 rclone config create myremote swift env_auth true 4171 rclone config create myremote swift env_auth=true 4172 \f[R] 4173 .fi 4174 .PP 4175 So for example if you wanted to configure a Google Drive remote but 4176 using remote authorization you would do this: 4177 .IP 4178 .nf 4179 \f[C] 4180 rclone config create mydrive drive config_is_local=false 4181 \f[R] 4182 .fi 4183 .PP 4184 Note that if the config process would normally ask a question the 4185 default is taken (unless \f[C]--non-interactive\f[R] is used). 4186 Each time that happens rclone will print or DEBUG a message saying how 4187 to affect the value taken. 4188 .PP 4189 If any of the parameters passed is a password field, then rclone will 4190 automatically obscure them if they aren\[aq]t already obscured before 4191 putting them in the config file. 4192 .PP 4193 \f[B]NB\f[R] If the password parameter is 22 characters or longer and 4194 consists only of base64 characters then rclone can get confused about 4195 whether the password is already obscured or not and put unobscured 4196 passwords into the config file. 4197 If you want to be 100% certain that the passwords get obscured then use 4198 the \f[C]--obscure\f[R] flag, or if you are 100% certain you are already 4199 passing obscured passwords then use \f[C]--no-obscure\f[R]. 4200 You can also set obscured passwords using the 4201 \f[C]rclone config password\f[R] command. 4202 .PP 4203 The flag \f[C]--non-interactive\f[R] is for use by applications that 4204 wish to configure rclone themselves, rather than using rclone\[aq]s text 4205 based configuration questions. 4206 If this flag is set, and rclone needs to ask the user a question, a JSON 4207 blob will be returned with the question in it. 4208 .PP 4209 This will look something like (some irrelevant detail removed): 4210 .IP 4211 .nf 4212 \f[C] 4213 { 4214 \[dq]State\[dq]: \[dq]*oauth-islocal,teamdrive,,\[dq], 4215 \[dq]Option\[dq]: { 4216 \[dq]Name\[dq]: \[dq]config_is_local\[dq], 4217 \[dq]Help\[dq]: \[dq]Use web browser to automatically authenticate rclone with remote?\[rs]n * Say Y if the machine running rclone has a web browser you can use\[rs]n * Say N if running rclone on a (remote) machine without web browser access\[rs]nIf not sure try Y. If Y failed, try N.\[rs]n\[dq], 4218 \[dq]Default\[dq]: true, 4219 \[dq]Examples\[dq]: [ 4220 { 4221 \[dq]Value\[dq]: \[dq]true\[dq], 4222 \[dq]Help\[dq]: \[dq]Yes\[dq] 4223 }, 4224 { 4225 \[dq]Value\[dq]: \[dq]false\[dq], 4226 \[dq]Help\[dq]: \[dq]No\[dq] 4227 } 4228 ], 4229 \[dq]Required\[dq]: false, 4230 \[dq]IsPassword\[dq]: false, 4231 \[dq]Type\[dq]: \[dq]bool\[dq], 4232 \[dq]Exclusive\[dq]: true, 4233 }, 4234 \[dq]Error\[dq]: \[dq]\[dq], 4235 } 4236 \f[R] 4237 .fi 4238 .PP 4239 The format of \f[C]Option\f[R] is the same as returned by 4240 \f[C]rclone config providers\f[R]. 4241 The question should be asked to the user and returned to rclone as the 4242 \f[C]--result\f[R] option along with the \f[C]--state\f[R] parameter. 4243 .PP 4244 The keys of \f[C]Option\f[R] are used as follows: 4245 .IP \[bu] 2 4246 \f[C]Name\f[R] - name of variable - show to user 4247 .IP \[bu] 2 4248 \f[C]Help\f[R] - help text. 4249 Hard wrapped at 80 chars. 4250 Any URLs should be clicky. 4251 .IP \[bu] 2 4252 \f[C]Default\f[R] - default value - return this if the user just wants 4253 the default. 4254 .IP \[bu] 2 4255 \f[C]Examples\f[R] - the user should be able to choose one of these 4256 .IP \[bu] 2 4257 \f[C]Required\f[R] - the value should be non-empty 4258 .IP \[bu] 2 4259 \f[C]IsPassword\f[R] - the value is a password and should be edited as 4260 such 4261 .IP \[bu] 2 4262 \f[C]Type\f[R] - type of value, eg \f[C]bool\f[R], \f[C]string\f[R], 4263 \f[C]int\f[R] and others 4264 .IP \[bu] 2 4265 \f[C]Exclusive\f[R] - if set no free-form entry allowed only the 4266 \f[C]Examples\f[R] 4267 .IP \[bu] 2 4268 Irrelevant keys \f[C]Provider\f[R], \f[C]ShortOpt\f[R], \f[C]Hide\f[R], 4269 \f[C]NoPrefix\f[R], \f[C]Advanced\f[R] 4270 .PP 4271 If \f[C]Error\f[R] is set then it should be shown to the user at the 4272 same time as the question. 4273 .IP 4274 .nf 4275 \f[C] 4276 rclone config update name --continue --state \[dq]*oauth-islocal,teamdrive,,\[dq] --result \[dq]true\[dq] 4277 \f[R] 4278 .fi 4279 .PP 4280 Note that when using \f[C]--continue\f[R] all passwords should be passed 4281 in the clear (not obscured). 4282 Any default config values should be passed in with each invocation of 4283 \f[C]--continue\f[R]. 4284 .PP 4285 At the end of the non interactive process, rclone will return a result 4286 with \f[C]State\f[R] as empty string. 4287 .PP 4288 If \f[C]--all\f[R] is passed then rclone will ask all the config 4289 questions, not just the post config questions. 4290 Any parameters are used as defaults for questions as usual. 4291 .PP 4292 Note that \f[C]bin/config.py\f[R] in the rclone source implements this 4293 protocol as a readable demonstration. 4294 .IP 4295 .nf 4296 \f[C] 4297 rclone config create name type [key value]* [flags] 4298 \f[R] 4299 .fi 4300 .SS Options 4301 .IP 4302 .nf 4303 \f[C] 4304 --all Ask the full set of config questions 4305 --continue Continue the configuration process with an answer 4306 -h, --help help for create 4307 --no-obscure Force any passwords not to be obscured 4308 --non-interactive Don\[aq]t interact with user and return questions 4309 --obscure Force any passwords to be obscured 4310 --result string Result - use with --continue 4311 --state string State - use with --continue 4312 \f[R] 4313 .fi 4314 .PP 4315 See the global flags page (https://rclone.org/flags/) for global options 4316 not listed here. 4317 .SH SEE ALSO 4318 .IP \[bu] 2 4319 rclone config (https://rclone.org/commands/rclone_config/) - Enter an 4320 interactive configuration session. 4321 .SH rclone config delete 4322 .PP 4323 Delete an existing remote. 4324 .IP 4325 .nf 4326 \f[C] 4327 rclone config delete name [flags] 4328 \f[R] 4329 .fi 4330 .SS Options 4331 .IP 4332 .nf 4333 \f[C] 4334 -h, --help help for delete 4335 \f[R] 4336 .fi 4337 .PP 4338 See the global flags page (https://rclone.org/flags/) for global options 4339 not listed here. 4340 .SH SEE ALSO 4341 .IP \[bu] 2 4342 rclone config (https://rclone.org/commands/rclone_config/) - Enter an 4343 interactive configuration session. 4344 .SH rclone config disconnect 4345 .PP 4346 Disconnects user from remote 4347 .SS Synopsis 4348 .PP 4349 This disconnects the remote: passed in to the cloud storage system. 4350 .PP 4351 This normally means revoking the oauth token. 4352 .PP 4353 To reconnect use \[dq]rclone config reconnect\[dq]. 4354 .IP 4355 .nf 4356 \f[C] 4357 rclone config disconnect remote: [flags] 4358 \f[R] 4359 .fi 4360 .SS Options 4361 .IP 4362 .nf 4363 \f[C] 4364 -h, --help help for disconnect 4365 \f[R] 4366 .fi 4367 .PP 4368 See the global flags page (https://rclone.org/flags/) for global options 4369 not listed here. 4370 .SH SEE ALSO 4371 .IP \[bu] 2 4372 rclone config (https://rclone.org/commands/rclone_config/) - Enter an 4373 interactive configuration session. 4374 .SH rclone config dump 4375 .PP 4376 Dump the config file as JSON. 4377 .IP 4378 .nf 4379 \f[C] 4380 rclone config dump [flags] 4381 \f[R] 4382 .fi 4383 .SS Options 4384 .IP 4385 .nf 4386 \f[C] 4387 -h, --help help for dump 4388 \f[R] 4389 .fi 4390 .PP 4391 See the global flags page (https://rclone.org/flags/) for global options 4392 not listed here. 4393 .SH SEE ALSO 4394 .IP \[bu] 2 4395 rclone config (https://rclone.org/commands/rclone_config/) - Enter an 4396 interactive configuration session. 4397 .SH rclone config edit 4398 .PP 4399 Enter an interactive configuration session. 4400 .SS Synopsis 4401 .PP 4402 Enter an interactive configuration session where you can setup new 4403 remotes and manage existing ones. 4404 You may also set or remove a password to protect your configuration. 4405 .IP 4406 .nf 4407 \f[C] 4408 rclone config edit [flags] 4409 \f[R] 4410 .fi 4411 .SS Options 4412 .IP 4413 .nf 4414 \f[C] 4415 -h, --help help for edit 4416 \f[R] 4417 .fi 4418 .PP 4419 See the global flags page (https://rclone.org/flags/) for global options 4420 not listed here. 4421 .SH SEE ALSO 4422 .IP \[bu] 2 4423 rclone config (https://rclone.org/commands/rclone_config/) - Enter an 4424 interactive configuration session. 4425 .SH rclone config file 4426 .PP 4427 Show path of configuration file in use. 4428 .IP 4429 .nf 4430 \f[C] 4431 rclone config file [flags] 4432 \f[R] 4433 .fi 4434 .SS Options 4435 .IP 4436 .nf 4437 \f[C] 4438 -h, --help help for file 4439 \f[R] 4440 .fi 4441 .PP 4442 See the global flags page (https://rclone.org/flags/) for global options 4443 not listed here. 4444 .SH SEE ALSO 4445 .IP \[bu] 2 4446 rclone config (https://rclone.org/commands/rclone_config/) - Enter an 4447 interactive configuration session. 4448 .SH rclone config password 4449 .PP 4450 Update password in an existing remote. 4451 .SS Synopsis 4452 .PP 4453 Update an existing remote\[aq]s password. 4454 The password should be passed in pairs of \f[C]key\f[R] 4455 \f[C]password\f[R] or as \f[C]key=password\f[R]. 4456 The \f[C]password\f[R] should be passed in in clear (unobscured). 4457 .PP 4458 For example, to set password of a remote of name myremote you would do: 4459 .IP 4460 .nf 4461 \f[C] 4462 rclone config password myremote fieldname mypassword 4463 rclone config password myremote fieldname=mypassword 4464 \f[R] 4465 .fi 4466 .PP 4467 This command is obsolete now that \[dq]config update\[dq] and 4468 \[dq]config create\[dq] both support obscuring passwords directly. 4469 .IP 4470 .nf 4471 \f[C] 4472 rclone config password name [key value]+ [flags] 4473 \f[R] 4474 .fi 4475 .SS Options 4476 .IP 4477 .nf 4478 \f[C] 4479 -h, --help help for password 4480 \f[R] 4481 .fi 4482 .PP 4483 See the global flags page (https://rclone.org/flags/) for global options 4484 not listed here. 4485 .SH SEE ALSO 4486 .IP \[bu] 2 4487 rclone config (https://rclone.org/commands/rclone_config/) - Enter an 4488 interactive configuration session. 4489 .SH rclone config paths 4490 .PP 4491 Show paths used for configuration, cache, temp etc. 4492 .IP 4493 .nf 4494 \f[C] 4495 rclone config paths [flags] 4496 \f[R] 4497 .fi 4498 .SS Options 4499 .IP 4500 .nf 4501 \f[C] 4502 -h, --help help for paths 4503 \f[R] 4504 .fi 4505 .PP 4506 See the global flags page (https://rclone.org/flags/) for global options 4507 not listed here. 4508 .SH SEE ALSO 4509 .IP \[bu] 2 4510 rclone config (https://rclone.org/commands/rclone_config/) - Enter an 4511 interactive configuration session. 4512 .SH rclone config providers 4513 .PP 4514 List in JSON format all the providers and options. 4515 .IP 4516 .nf 4517 \f[C] 4518 rclone config providers [flags] 4519 \f[R] 4520 .fi 4521 .SS Options 4522 .IP 4523 .nf 4524 \f[C] 4525 -h, --help help for providers 4526 \f[R] 4527 .fi 4528 .PP 4529 See the global flags page (https://rclone.org/flags/) for global options 4530 not listed here. 4531 .SH SEE ALSO 4532 .IP \[bu] 2 4533 rclone config (https://rclone.org/commands/rclone_config/) - Enter an 4534 interactive configuration session. 4535 .SH rclone config reconnect 4536 .PP 4537 Re-authenticates user with remote. 4538 .SS Synopsis 4539 .PP 4540 This reconnects remote: passed in to the cloud storage system. 4541 .PP 4542 To disconnect the remote use \[dq]rclone config disconnect\[dq]. 4543 .PP 4544 This normally means going through the interactive oauth flow again. 4545 .IP 4546 .nf 4547 \f[C] 4548 rclone config reconnect remote: [flags] 4549 \f[R] 4550 .fi 4551 .SS Options 4552 .IP 4553 .nf 4554 \f[C] 4555 -h, --help help for reconnect 4556 \f[R] 4557 .fi 4558 .PP 4559 See the global flags page (https://rclone.org/flags/) for global options 4560 not listed here. 4561 .SH SEE ALSO 4562 .IP \[bu] 2 4563 rclone config (https://rclone.org/commands/rclone_config/) - Enter an 4564 interactive configuration session. 4565 .SH rclone config redacted 4566 .PP 4567 Print redacted (decrypted) config file, or the redacted config for a 4568 single remote. 4569 .SS Synopsis 4570 .PP 4571 This prints a redacted copy of the config file, either the whole config 4572 file or for a given remote. 4573 .PP 4574 The config file will be redacted by replacing all passwords and other 4575 sensitive info with XXX. 4576 .PP 4577 This makes the config file suitable for posting online for support. 4578 .PP 4579 It should be double checked before posting as the redaction may not be 4580 perfect. 4581 .IP 4582 .nf 4583 \f[C] 4584 rclone config redacted [<remote>] [flags] 4585 \f[R] 4586 .fi 4587 .SS Options 4588 .IP 4589 .nf 4590 \f[C] 4591 -h, --help help for redacted 4592 \f[R] 4593 .fi 4594 .PP 4595 See the global flags page (https://rclone.org/flags/) for global options 4596 not listed here. 4597 .SH SEE ALSO 4598 .IP \[bu] 2 4599 rclone config (https://rclone.org/commands/rclone_config/) - Enter an 4600 interactive configuration session. 4601 .SH rclone config show 4602 .PP 4603 Print (decrypted) config file, or the config for a single remote. 4604 .IP 4605 .nf 4606 \f[C] 4607 rclone config show [<remote>] [flags] 4608 \f[R] 4609 .fi 4610 .SS Options 4611 .IP 4612 .nf 4613 \f[C] 4614 -h, --help help for show 4615 \f[R] 4616 .fi 4617 .PP 4618 See the global flags page (https://rclone.org/flags/) for global options 4619 not listed here. 4620 .SH SEE ALSO 4621 .IP \[bu] 2 4622 rclone config (https://rclone.org/commands/rclone_config/) - Enter an 4623 interactive configuration session. 4624 .SH rclone config touch 4625 .PP 4626 Ensure configuration file exists. 4627 .IP 4628 .nf 4629 \f[C] 4630 rclone config touch [flags] 4631 \f[R] 4632 .fi 4633 .SS Options 4634 .IP 4635 .nf 4636 \f[C] 4637 -h, --help help for touch 4638 \f[R] 4639 .fi 4640 .PP 4641 See the global flags page (https://rclone.org/flags/) for global options 4642 not listed here. 4643 .SH SEE ALSO 4644 .IP \[bu] 2 4645 rclone config (https://rclone.org/commands/rclone_config/) - Enter an 4646 interactive configuration session. 4647 .SH rclone config update 4648 .PP 4649 Update options in an existing remote. 4650 .SS Synopsis 4651 .PP 4652 Update an existing remote\[aq]s options. 4653 The options should be passed in pairs of \f[C]key\f[R] \f[C]value\f[R] 4654 or as \f[C]key=value\f[R]. 4655 .PP 4656 For example, to update the env_auth field of a remote of name myremote 4657 you would do: 4658 .IP 4659 .nf 4660 \f[C] 4661 rclone config update myremote env_auth true 4662 rclone config update myremote env_auth=true 4663 \f[R] 4664 .fi 4665 .PP 4666 If the remote uses OAuth the token will be updated, if you don\[aq]t 4667 require this add an extra parameter thus: 4668 .IP 4669 .nf 4670 \f[C] 4671 rclone config update myremote env_auth=true config_refresh_token=false 4672 \f[R] 4673 .fi 4674 .PP 4675 Note that if the config process would normally ask a question the 4676 default is taken (unless \f[C]--non-interactive\f[R] is used). 4677 Each time that happens rclone will print or DEBUG a message saying how 4678 to affect the value taken. 4679 .PP 4680 If any of the parameters passed is a password field, then rclone will 4681 automatically obscure them if they aren\[aq]t already obscured before 4682 putting them in the config file. 4683 .PP 4684 \f[B]NB\f[R] If the password parameter is 22 characters or longer and 4685 consists only of base64 characters then rclone can get confused about 4686 whether the password is already obscured or not and put unobscured 4687 passwords into the config file. 4688 If you want to be 100% certain that the passwords get obscured then use 4689 the \f[C]--obscure\f[R] flag, or if you are 100% certain you are already 4690 passing obscured passwords then use \f[C]--no-obscure\f[R]. 4691 You can also set obscured passwords using the 4692 \f[C]rclone config password\f[R] command. 4693 .PP 4694 The flag \f[C]--non-interactive\f[R] is for use by applications that 4695 wish to configure rclone themselves, rather than using rclone\[aq]s text 4696 based configuration questions. 4697 If this flag is set, and rclone needs to ask the user a question, a JSON 4698 blob will be returned with the question in it. 4699 .PP 4700 This will look something like (some irrelevant detail removed): 4701 .IP 4702 .nf 4703 \f[C] 4704 { 4705 \[dq]State\[dq]: \[dq]*oauth-islocal,teamdrive,,\[dq], 4706 \[dq]Option\[dq]: { 4707 \[dq]Name\[dq]: \[dq]config_is_local\[dq], 4708 \[dq]Help\[dq]: \[dq]Use web browser to automatically authenticate rclone with remote?\[rs]n * Say Y if the machine running rclone has a web browser you can use\[rs]n * Say N if running rclone on a (remote) machine without web browser access\[rs]nIf not sure try Y. If Y failed, try N.\[rs]n\[dq], 4709 \[dq]Default\[dq]: true, 4710 \[dq]Examples\[dq]: [ 4711 { 4712 \[dq]Value\[dq]: \[dq]true\[dq], 4713 \[dq]Help\[dq]: \[dq]Yes\[dq] 4714 }, 4715 { 4716 \[dq]Value\[dq]: \[dq]false\[dq], 4717 \[dq]Help\[dq]: \[dq]No\[dq] 4718 } 4719 ], 4720 \[dq]Required\[dq]: false, 4721 \[dq]IsPassword\[dq]: false, 4722 \[dq]Type\[dq]: \[dq]bool\[dq], 4723 \[dq]Exclusive\[dq]: true, 4724 }, 4725 \[dq]Error\[dq]: \[dq]\[dq], 4726 } 4727 \f[R] 4728 .fi 4729 .PP 4730 The format of \f[C]Option\f[R] is the same as returned by 4731 \f[C]rclone config providers\f[R]. 4732 The question should be asked to the user and returned to rclone as the 4733 \f[C]--result\f[R] option along with the \f[C]--state\f[R] parameter. 4734 .PP 4735 The keys of \f[C]Option\f[R] are used as follows: 4736 .IP \[bu] 2 4737 \f[C]Name\f[R] - name of variable - show to user 4738 .IP \[bu] 2 4739 \f[C]Help\f[R] - help text. 4740 Hard wrapped at 80 chars. 4741 Any URLs should be clicky. 4742 .IP \[bu] 2 4743 \f[C]Default\f[R] - default value - return this if the user just wants 4744 the default. 4745 .IP \[bu] 2 4746 \f[C]Examples\f[R] - the user should be able to choose one of these 4747 .IP \[bu] 2 4748 \f[C]Required\f[R] - the value should be non-empty 4749 .IP \[bu] 2 4750 \f[C]IsPassword\f[R] - the value is a password and should be edited as 4751 such 4752 .IP \[bu] 2 4753 \f[C]Type\f[R] - type of value, eg \f[C]bool\f[R], \f[C]string\f[R], 4754 \f[C]int\f[R] and others 4755 .IP \[bu] 2 4756 \f[C]Exclusive\f[R] - if set no free-form entry allowed only the 4757 \f[C]Examples\f[R] 4758 .IP \[bu] 2 4759 Irrelevant keys \f[C]Provider\f[R], \f[C]ShortOpt\f[R], \f[C]Hide\f[R], 4760 \f[C]NoPrefix\f[R], \f[C]Advanced\f[R] 4761 .PP 4762 If \f[C]Error\f[R] is set then it should be shown to the user at the 4763 same time as the question. 4764 .IP 4765 .nf 4766 \f[C] 4767 rclone config update name --continue --state \[dq]*oauth-islocal,teamdrive,,\[dq] --result \[dq]true\[dq] 4768 \f[R] 4769 .fi 4770 .PP 4771 Note that when using \f[C]--continue\f[R] all passwords should be passed 4772 in the clear (not obscured). 4773 Any default config values should be passed in with each invocation of 4774 \f[C]--continue\f[R]. 4775 .PP 4776 At the end of the non interactive process, rclone will return a result 4777 with \f[C]State\f[R] as empty string. 4778 .PP 4779 If \f[C]--all\f[R] is passed then rclone will ask all the config 4780 questions, not just the post config questions. 4781 Any parameters are used as defaults for questions as usual. 4782 .PP 4783 Note that \f[C]bin/config.py\f[R] in the rclone source implements this 4784 protocol as a readable demonstration. 4785 .IP 4786 .nf 4787 \f[C] 4788 rclone config update name [key value]+ [flags] 4789 \f[R] 4790 .fi 4791 .SS Options 4792 .IP 4793 .nf 4794 \f[C] 4795 --all Ask the full set of config questions 4796 --continue Continue the configuration process with an answer 4797 -h, --help help for update 4798 --no-obscure Force any passwords not to be obscured 4799 --non-interactive Don\[aq]t interact with user and return questions 4800 --obscure Force any passwords to be obscured 4801 --result string Result - use with --continue 4802 --state string State - use with --continue 4803 \f[R] 4804 .fi 4805 .PP 4806 See the global flags page (https://rclone.org/flags/) for global options 4807 not listed here. 4808 .SH SEE ALSO 4809 .IP \[bu] 2 4810 rclone config (https://rclone.org/commands/rclone_config/) - Enter an 4811 interactive configuration session. 4812 .SH rclone config userinfo 4813 .PP 4814 Prints info about logged in user of remote. 4815 .SS Synopsis 4816 .PP 4817 This prints the details of the person logged in to the cloud storage 4818 system. 4819 .IP 4820 .nf 4821 \f[C] 4822 rclone config userinfo remote: [flags] 4823 \f[R] 4824 .fi 4825 .SS Options 4826 .IP 4827 .nf 4828 \f[C] 4829 -h, --help help for userinfo 4830 --json Format output as JSON 4831 \f[R] 4832 .fi 4833 .PP 4834 See the global flags page (https://rclone.org/flags/) for global options 4835 not listed here. 4836 .SH SEE ALSO 4837 .IP \[bu] 2 4838 rclone config (https://rclone.org/commands/rclone_config/) - Enter an 4839 interactive configuration session. 4840 .SH rclone copyto 4841 .PP 4842 Copy files from source to dest, skipping identical files. 4843 .SS Synopsis 4844 .PP 4845 If source:path is a file or directory then it copies it to a file or 4846 directory named dest:path. 4847 .PP 4848 This can be used to upload single files to other than their current 4849 name. 4850 If the source is a directory then it acts exactly like the 4851 copy (https://rclone.org/commands/rclone_copy/) command. 4852 .PP 4853 So 4854 .IP 4855 .nf 4856 \f[C] 4857 rclone copyto src dst 4858 \f[R] 4859 .fi 4860 .PP 4861 where src and dst are rclone paths, either remote:path or /path/to/local 4862 or C:. 4863 .PP 4864 This will: 4865 .IP 4866 .nf 4867 \f[C] 4868 if src is file 4869 copy it to dst, overwriting an existing file if it exists 4870 if src is directory 4871 copy it to dst, overwriting existing files if they exist 4872 see copy command for full details 4873 \f[R] 4874 .fi 4875 .PP 4876 This doesn\[aq]t transfer files that are identical on src and dst, 4877 testing by size and modification time or MD5SUM. 4878 It doesn\[aq]t delete files from the destination. 4879 .PP 4880 \f[B]Note\f[R]: Use the \f[C]-P\f[R]/\f[C]--progress\f[R] flag to view 4881 real-time transfer statistics 4882 .IP 4883 .nf 4884 \f[C] 4885 rclone copyto source:path dest:path [flags] 4886 \f[R] 4887 .fi 4888 .SS Options 4889 .IP 4890 .nf 4891 \f[C] 4892 -h, --help help for copyto 4893 \f[R] 4894 .fi 4895 .SS Copy Options 4896 .PP 4897 Flags for anything which can Copy a file. 4898 .IP 4899 .nf 4900 \f[C] 4901 --check-first Do all the checks before starting transfers 4902 -c, --checksum Check for changes with size & checksum (if available, or fallback to size only). 4903 --compare-dest stringArray Include additional comma separated server-side paths during comparison 4904 --copy-dest stringArray Implies --compare-dest but also copies files from paths into destination 4905 --cutoff-mode HARD|SOFT|CAUTIOUS Mode to stop transfers when reaching the max transfer limit HARD|SOFT|CAUTIOUS (default HARD) 4906 --ignore-case-sync Ignore case when synchronizing 4907 --ignore-checksum Skip post copy check of checksums 4908 --ignore-existing Skip all files that exist on destination 4909 --ignore-size Ignore size when skipping use modtime or checksum 4910 -I, --ignore-times Don\[aq]t skip items that match size and time - transfer all unconditionally 4911 --immutable Do not modify files, fail if existing files have been modified 4912 --inplace Download directly to destination file instead of atomic download to temp/rename 4913 --max-backlog int Maximum number of objects in sync or check backlog (default 10000) 4914 --max-duration Duration Maximum duration rclone will transfer data for (default 0s) 4915 --max-transfer SizeSuffix Maximum size of data to transfer (default off) 4916 -M, --metadata If set, preserve metadata when copying objects 4917 --modify-window Duration Max time diff to be considered the same (default 1ns) 4918 --multi-thread-chunk-size SizeSuffix Chunk size for multi-thread downloads / uploads, if not set by filesystem (default 64Mi) 4919 --multi-thread-cutoff SizeSuffix Use multi-thread downloads for files above this size (default 256Mi) 4920 --multi-thread-streams int Number of streams to use for multi-thread downloads (default 4) 4921 --multi-thread-write-buffer-size SizeSuffix In memory buffer size for writing when in multi-thread mode (default 128Ki) 4922 --no-check-dest Don\[aq]t check the destination, copy regardless 4923 --no-traverse Don\[aq]t traverse destination file system on copy 4924 --no-update-dir-modtime Don\[aq]t update directory modification times 4925 --no-update-modtime Don\[aq]t update destination modtime if files identical 4926 --order-by string Instructions on how to order the transfers, e.g. \[aq]size,descending\[aq] 4927 --partial-suffix string Add partial-suffix to temporary file name when --inplace is not used (default \[dq].partial\[dq]) 4928 --refresh-times Refresh the modtime of remote files 4929 --server-side-across-configs Allow server-side operations (e.g. copy) to work across different configs 4930 --size-only Skip based on size only, not modtime or checksum 4931 --streaming-upload-cutoff SizeSuffix Cutoff for switching to chunked upload if file size is unknown, upload starts after reaching cutoff or when file ends (default 100Ki) 4932 -u, --update Skip files that are newer on the destination 4933 \f[R] 4934 .fi 4935 .SS Important Options 4936 .PP 4937 Important flags useful for most commands. 4938 .IP 4939 .nf 4940 \f[C] 4941 -n, --dry-run Do a trial run with no permanent changes 4942 -i, --interactive Enable interactive mode 4943 -v, --verbose count Print lots more stuff (repeat for more) 4944 \f[R] 4945 .fi 4946 .SS Filter Options 4947 .PP 4948 Flags for filtering directory listings. 4949 .IP 4950 .nf 4951 \f[C] 4952 --delete-excluded Delete files on dest excluded from sync 4953 --exclude stringArray Exclude files matching pattern 4954 --exclude-from stringArray Read file exclude patterns from file (use - to read from stdin) 4955 --exclude-if-present stringArray Exclude directories if filename is present 4956 --files-from stringArray Read list of source-file names from file (use - to read from stdin) 4957 --files-from-raw stringArray Read list of source-file names from file without any processing of lines (use - to read from stdin) 4958 -f, --filter stringArray Add a file filtering rule 4959 --filter-from stringArray Read file filtering patterns from a file (use - to read from stdin) 4960 --ignore-case Ignore case in filters (case insensitive) 4961 --include stringArray Include files matching pattern 4962 --include-from stringArray Read file include patterns from file (use - to read from stdin) 4963 --max-age Duration Only transfer files younger than this in s or suffix ms|s|m|h|d|w|M|y (default off) 4964 --max-depth int If set limits the recursion depth to this (default -1) 4965 --max-size SizeSuffix Only transfer files smaller than this in KiB or suffix B|K|M|G|T|P (default off) 4966 --metadata-exclude stringArray Exclude metadatas matching pattern 4967 --metadata-exclude-from stringArray Read metadata exclude patterns from file (use - to read from stdin) 4968 --metadata-filter stringArray Add a metadata filtering rule 4969 --metadata-filter-from stringArray Read metadata filtering patterns from a file (use - to read from stdin) 4970 --metadata-include stringArray Include metadatas matching pattern 4971 --metadata-include-from stringArray Read metadata include patterns from file (use - to read from stdin) 4972 --min-age Duration Only transfer files older than this in s or suffix ms|s|m|h|d|w|M|y (default off) 4973 --min-size SizeSuffix Only transfer files bigger than this in KiB or suffix B|K|M|G|T|P (default off) 4974 \f[R] 4975 .fi 4976 .SS Listing Options 4977 .PP 4978 Flags for listing directories. 4979 .IP 4980 .nf 4981 \f[C] 4982 --default-time Time Time to show if modtime is unknown for files and directories (default 2000-01-01T00:00:00Z) 4983 --fast-list Use recursive list if available; uses more memory but fewer transactions 4984 \f[R] 4985 .fi 4986 .PP 4987 See the global flags page (https://rclone.org/flags/) for global options 4988 not listed here. 4989 .SH SEE ALSO 4990 .IP \[bu] 2 4991 rclone (https://rclone.org/commands/rclone/) - Show help for rclone 4992 commands, flags and backends. 4993 .SH rclone copyurl 4994 .PP 4995 Copy the contents of the URL supplied content to dest:path. 4996 .SS Synopsis 4997 .PP 4998 Download a URL\[aq]s content and copy it to the destination without 4999 saving it in temporary storage. 5000 .PP 5001 Setting \f[C]--auto-filename\f[R] will attempt to automatically 5002 determine the filename from the URL (after any redirections) and used in 5003 the destination path. 5004 .PP 5005 With \f[C]--auto-filename-header\f[R] in addition, if a specific 5006 filename is set in HTTP headers, it will be used instead of the name 5007 from the URL. 5008 With \f[C]--print-filename\f[R] in addition, the resulting file name 5009 will be printed. 5010 .PP 5011 Setting \f[C]--no-clobber\f[R] will prevent overwriting file on the 5012 destination if there is one with the same name. 5013 .PP 5014 Setting \f[C]--stdout\f[R] or making the output file name \f[C]-\f[R] 5015 will cause the output to be written to standard output. 5016 .SS Troublshooting 5017 .PP 5018 If you can\[aq]t get \f[C]rclone copyurl\f[R] to work then here are some 5019 things you can try: 5020 .IP \[bu] 2 5021 \f[C]--disable-http2\f[R] rclone will use HTTP2 if available - try 5022 disabling it 5023 .IP \[bu] 2 5024 \f[C]--bind 0.0.0.0\f[R] rclone will use IPv6 if available - try 5025 disabling it 5026 .IP \[bu] 2 5027 \f[C]--bind ::0\f[R] to disable IPv4 5028 .IP \[bu] 2 5029 \f[C]--user agent curl\f[R] - some sites have whitelists for curl\[aq]s 5030 user-agent - try that 5031 .IP \[bu] 2 5032 Make sure the site works with \f[C]curl\f[R] directly 5033 .IP 5034 .nf 5035 \f[C] 5036 rclone copyurl https://example.com dest:path [flags] 5037 \f[R] 5038 .fi 5039 .SS Options 5040 .IP 5041 .nf 5042 \f[C] 5043 -a, --auto-filename Get the file name from the URL and use it for destination file path 5044 --header-filename Get the file name from the Content-Disposition header 5045 -h, --help help for copyurl 5046 --no-clobber Prevent overwriting file with same name 5047 -p, --print-filename Print the resulting name from --auto-filename 5048 --stdout Write the output to stdout rather than a file 5049 \f[R] 5050 .fi 5051 .SS Important Options 5052 .PP 5053 Important flags useful for most commands. 5054 .IP 5055 .nf 5056 \f[C] 5057 -n, --dry-run Do a trial run with no permanent changes 5058 -i, --interactive Enable interactive mode 5059 -v, --verbose count Print lots more stuff (repeat for more) 5060 \f[R] 5061 .fi 5062 .PP 5063 See the global flags page (https://rclone.org/flags/) for global options 5064 not listed here. 5065 .SH SEE ALSO 5066 .IP \[bu] 2 5067 rclone (https://rclone.org/commands/rclone/) - Show help for rclone 5068 commands, flags and backends. 5069 .SH rclone cryptcheck 5070 .PP 5071 Cryptcheck checks the integrity of an encrypted remote. 5072 .SS Synopsis 5073 .PP 5074 rclone cryptcheck checks a remote against a 5075 crypted (https://rclone.org/crypt/) remote. 5076 This is the equivalent of running rclone 5077 check (https://rclone.org/commands/rclone_check/), but able to check the 5078 checksums of the encrypted remote. 5079 .PP 5080 For it to work the underlying remote of the cryptedremote must support 5081 some kind of checksum. 5082 .PP 5083 It works by reading the nonce from each file on the cryptedremote: and 5084 using that to encrypt each file on the remote:. 5085 It then checks the checksum of the underlying file on the cryptedremote: 5086 against the checksum of the file it has just encrypted. 5087 .PP 5088 Use it like this 5089 .IP 5090 .nf 5091 \f[C] 5092 rclone cryptcheck /path/to/files encryptedremote:path 5093 \f[R] 5094 .fi 5095 .PP 5096 You can use it like this also, but that will involve downloading all the 5097 files in remote:path. 5098 .IP 5099 .nf 5100 \f[C] 5101 rclone cryptcheck remote:path encryptedremote:path 5102 \f[R] 5103 .fi 5104 .PP 5105 After it has run it will log the status of the encryptedremote:. 5106 .PP 5107 If you supply the \f[C]--one-way\f[R] flag, it will only check that 5108 files in the source match the files in the destination, not the other 5109 way around. 5110 This means that extra files in the destination that are not in the 5111 source will not be detected. 5112 .PP 5113 The \f[C]--differ\f[R], \f[C]--missing-on-dst\f[R], 5114 \f[C]--missing-on-src\f[R], \f[C]--match\f[R] and \f[C]--error\f[R] 5115 flags write paths, one per line, to the file name (or stdout if it is 5116 \f[C]-\f[R]) supplied. 5117 What they write is described in the help below. 5118 For example \f[C]--differ\f[R] will write all paths which are present on 5119 both the source and destination but different. 5120 .PP 5121 The \f[C]--combined\f[R] flag will write a file (or stdout) which 5122 contains all file paths with a symbol and then a space and then the path 5123 to tell you what happened to it. 5124 These are reminiscent of diff files. 5125 .IP \[bu] 2 5126 \f[C]= path\f[R] means path was found in source and destination and was 5127 identical 5128 .IP \[bu] 2 5129 \[ga]- path\[ga] means path was missing on the source, so only in the 5130 destination 5131 .IP \[bu] 2 5132 \[ga]+ path\[ga] means path was missing on the destination, so only in 5133 the source 5134 .IP \[bu] 2 5135 \[ga]* path\[ga] means path was present in source and destination but 5136 different. 5137 .IP \[bu] 2 5138 \f[C]! path\f[R] means there was an error reading or hashing the source 5139 or dest. 5140 .PP 5141 The default number of parallel checks is 8. 5142 See the --checkers=N (https://rclone.org/docs/#checkers-n) option for 5143 more information. 5144 .IP 5145 .nf 5146 \f[C] 5147 rclone cryptcheck remote:path cryptedremote:path [flags] 5148 \f[R] 5149 .fi 5150 .SS Options 5151 .IP 5152 .nf 5153 \f[C] 5154 --combined string Make a combined report of changes to this file 5155 --differ string Report all non-matching files to this file 5156 --error string Report all files with errors (hashing or reading) to this file 5157 -h, --help help for cryptcheck 5158 --match string Report all matching files to this file 5159 --missing-on-dst string Report all files missing from the destination to this file 5160 --missing-on-src string Report all files missing from the source to this file 5161 --one-way Check one way only, source files must exist on remote 5162 \f[R] 5163 .fi 5164 .SS Check Options 5165 .PP 5166 Flags used for \f[C]rclone check\f[R]. 5167 .IP 5168 .nf 5169 \f[C] 5170 --max-backlog int Maximum number of objects in sync or check backlog (default 10000) 5171 \f[R] 5172 .fi 5173 .SS Filter Options 5174 .PP 5175 Flags for filtering directory listings. 5176 .IP 5177 .nf 5178 \f[C] 5179 --delete-excluded Delete files on dest excluded from sync 5180 --exclude stringArray Exclude files matching pattern 5181 --exclude-from stringArray Read file exclude patterns from file (use - to read from stdin) 5182 --exclude-if-present stringArray Exclude directories if filename is present 5183 --files-from stringArray Read list of source-file names from file (use - to read from stdin) 5184 --files-from-raw stringArray Read list of source-file names from file without any processing of lines (use - to read from stdin) 5185 -f, --filter stringArray Add a file filtering rule 5186 --filter-from stringArray Read file filtering patterns from a file (use - to read from stdin) 5187 --ignore-case Ignore case in filters (case insensitive) 5188 --include stringArray Include files matching pattern 5189 --include-from stringArray Read file include patterns from file (use - to read from stdin) 5190 --max-age Duration Only transfer files younger than this in s or suffix ms|s|m|h|d|w|M|y (default off) 5191 --max-depth int If set limits the recursion depth to this (default -1) 5192 --max-size SizeSuffix Only transfer files smaller than this in KiB or suffix B|K|M|G|T|P (default off) 5193 --metadata-exclude stringArray Exclude metadatas matching pattern 5194 --metadata-exclude-from stringArray Read metadata exclude patterns from file (use - to read from stdin) 5195 --metadata-filter stringArray Add a metadata filtering rule 5196 --metadata-filter-from stringArray Read metadata filtering patterns from a file (use - to read from stdin) 5197 --metadata-include stringArray Include metadatas matching pattern 5198 --metadata-include-from stringArray Read metadata include patterns from file (use - to read from stdin) 5199 --min-age Duration Only transfer files older than this in s or suffix ms|s|m|h|d|w|M|y (default off) 5200 --min-size SizeSuffix Only transfer files bigger than this in KiB or suffix B|K|M|G|T|P (default off) 5201 \f[R] 5202 .fi 5203 .SS Listing Options 5204 .PP 5205 Flags for listing directories. 5206 .IP 5207 .nf 5208 \f[C] 5209 --default-time Time Time to show if modtime is unknown for files and directories (default 2000-01-01T00:00:00Z) 5210 --fast-list Use recursive list if available; uses more memory but fewer transactions 5211 \f[R] 5212 .fi 5213 .PP 5214 See the global flags page (https://rclone.org/flags/) for global options 5215 not listed here. 5216 .SH SEE ALSO 5217 .IP \[bu] 2 5218 rclone (https://rclone.org/commands/rclone/) - Show help for rclone 5219 commands, flags and backends. 5220 .SH rclone cryptdecode 5221 .PP 5222 Cryptdecode returns unencrypted file names. 5223 .SS Synopsis 5224 .PP 5225 rclone cryptdecode returns unencrypted file names when provided with a 5226 list of encrypted file names. 5227 List limit is 10 items. 5228 .PP 5229 If you supply the \f[C]--reverse\f[R] flag, it will return encrypted 5230 file names. 5231 .PP 5232 use it like this 5233 .IP 5234 .nf 5235 \f[C] 5236 rclone cryptdecode encryptedremote: encryptedfilename1 encryptedfilename2 5237 5238 rclone cryptdecode --reverse encryptedremote: filename1 filename2 5239 \f[R] 5240 .fi 5241 .PP 5242 Another way to accomplish this is by using the 5243 \f[C]rclone backend encode\f[R] (or \f[C]decode\f[R]) command. 5244 See the documentation on the crypt (https://rclone.org/crypt/) overlay 5245 for more info. 5246 .IP 5247 .nf 5248 \f[C] 5249 rclone cryptdecode encryptedremote: encryptedfilename [flags] 5250 \f[R] 5251 .fi 5252 .SS Options 5253 .IP 5254 .nf 5255 \f[C] 5256 -h, --help help for cryptdecode 5257 --reverse Reverse cryptdecode, encrypts filenames 5258 \f[R] 5259 .fi 5260 .PP 5261 See the global flags page (https://rclone.org/flags/) for global options 5262 not listed here. 5263 .SH SEE ALSO 5264 .IP \[bu] 2 5265 rclone (https://rclone.org/commands/rclone/) - Show help for rclone 5266 commands, flags and backends. 5267 .SH rclone deletefile 5268 .PP 5269 Remove a single file from remote. 5270 .SS Synopsis 5271 .PP 5272 Remove a single file from remote. 5273 Unlike \f[C]delete\f[R] it cannot be used to remove a directory and it 5274 doesn\[aq]t obey include/exclude filters - if the specified file exists, 5275 it will always be removed. 5276 .IP 5277 .nf 5278 \f[C] 5279 rclone deletefile remote:path [flags] 5280 \f[R] 5281 .fi 5282 .SS Options 5283 .IP 5284 .nf 5285 \f[C] 5286 -h, --help help for deletefile 5287 \f[R] 5288 .fi 5289 .SS Important Options 5290 .PP 5291 Important flags useful for most commands. 5292 .IP 5293 .nf 5294 \f[C] 5295 -n, --dry-run Do a trial run with no permanent changes 5296 -i, --interactive Enable interactive mode 5297 -v, --verbose count Print lots more stuff (repeat for more) 5298 \f[R] 5299 .fi 5300 .PP 5301 See the global flags page (https://rclone.org/flags/) for global options 5302 not listed here. 5303 .SH SEE ALSO 5304 .IP \[bu] 2 5305 rclone (https://rclone.org/commands/rclone/) - Show help for rclone 5306 commands, flags and backends. 5307 .SH rclone genautocomplete 5308 .PP 5309 Output completion script for a given shell. 5310 .SH Synopsis 5311 .PP 5312 Generates a shell completion script for rclone. 5313 Run with \f[C]--help\f[R] to list the supported shells. 5314 .SH Options 5315 .IP 5316 .nf 5317 \f[C] 5318 -h, --help help for genautocomplete 5319 \f[R] 5320 .fi 5321 .PP 5322 See the global flags page (https://rclone.org/flags/) for global options 5323 not listed here. 5324 .SH SEE ALSO 5325 .IP \[bu] 2 5326 rclone (https://rclone.org/commands/rclone/) - Show help for rclone 5327 commands, flags and backends. 5328 .IP \[bu] 2 5329 rclone genautocomplete 5330 bash (https://rclone.org/commands/rclone_genautocomplete_bash/) - Output 5331 bash completion script for rclone. 5332 .IP \[bu] 2 5333 rclone genautocomplete 5334 fish (https://rclone.org/commands/rclone_genautocomplete_fish/) - Output 5335 fish completion script for rclone. 5336 .IP \[bu] 2 5337 rclone genautocomplete 5338 zsh (https://rclone.org/commands/rclone_genautocomplete_zsh/) - Output 5339 zsh completion script for rclone. 5340 .SH rclone genautocomplete bash 5341 .PP 5342 Output bash completion script for rclone. 5343 .SH Synopsis 5344 .PP 5345 Generates a bash shell autocompletion script for rclone. 5346 .PP 5347 This writes to /etc/bash_completion.d/rclone by default so will probably 5348 need to be run with sudo or as root, e.g. 5349 .IP 5350 .nf 5351 \f[C] 5352 sudo rclone genautocomplete bash 5353 \f[R] 5354 .fi 5355 .PP 5356 Logout and login again to use the autocompletion scripts, or source them 5357 directly 5358 .IP 5359 .nf 5360 \f[C] 5361 \&. /etc/bash_completion 5362 \f[R] 5363 .fi 5364 .PP 5365 If you supply a command line argument the script will be written there. 5366 .PP 5367 If output_file is \[dq]-\[dq], then the output will be written to 5368 stdout. 5369 .IP 5370 .nf 5371 \f[C] 5372 rclone genautocomplete bash [output_file] [flags] 5373 \f[R] 5374 .fi 5375 .SH Options 5376 .IP 5377 .nf 5378 \f[C] 5379 -h, --help help for bash 5380 \f[R] 5381 .fi 5382 .PP 5383 See the global flags page (https://rclone.org/flags/) for global options 5384 not listed here. 5385 .SH SEE ALSO 5386 .IP \[bu] 2 5387 rclone 5388 genautocomplete (https://rclone.org/commands/rclone_genautocomplete/) - 5389 Output completion script for a given shell. 5390 .SH rclone genautocomplete fish 5391 .PP 5392 Output fish completion script for rclone. 5393 .SH Synopsis 5394 .PP 5395 Generates a fish autocompletion script for rclone. 5396 .PP 5397 This writes to /etc/fish/completions/rclone.fish by default so will 5398 probably need to be run with sudo or as root, e.g. 5399 .IP 5400 .nf 5401 \f[C] 5402 sudo rclone genautocomplete fish 5403 \f[R] 5404 .fi 5405 .PP 5406 Logout and login again to use the autocompletion scripts, or source them 5407 directly 5408 .IP 5409 .nf 5410 \f[C] 5411 \&. /etc/fish/completions/rclone.fish 5412 \f[R] 5413 .fi 5414 .PP 5415 If you supply a command line argument the script will be written there. 5416 .PP 5417 If output_file is \[dq]-\[dq], then the output will be written to 5418 stdout. 5419 .IP 5420 .nf 5421 \f[C] 5422 rclone genautocomplete fish [output_file] [flags] 5423 \f[R] 5424 .fi 5425 .SH Options 5426 .IP 5427 .nf 5428 \f[C] 5429 -h, --help help for fish 5430 \f[R] 5431 .fi 5432 .PP 5433 See the global flags page (https://rclone.org/flags/) for global options 5434 not listed here. 5435 .SH SEE ALSO 5436 .IP \[bu] 2 5437 rclone 5438 genautocomplete (https://rclone.org/commands/rclone_genautocomplete/) - 5439 Output completion script for a given shell. 5440 .SH rclone genautocomplete zsh 5441 .PP 5442 Output zsh completion script for rclone. 5443 .SH Synopsis 5444 .PP 5445 Generates a zsh autocompletion script for rclone. 5446 .PP 5447 This writes to /usr/share/zsh/vendor-completions/_rclone by default so 5448 will probably need to be run with sudo or as root, e.g. 5449 .IP 5450 .nf 5451 \f[C] 5452 sudo rclone genautocomplete zsh 5453 \f[R] 5454 .fi 5455 .PP 5456 Logout and login again to use the autocompletion scripts, or source them 5457 directly 5458 .IP 5459 .nf 5460 \f[C] 5461 autoload -U compinit && compinit 5462 \f[R] 5463 .fi 5464 .PP 5465 If you supply a command line argument the script will be written there. 5466 .PP 5467 If output_file is \[dq]-\[dq], then the output will be written to 5468 stdout. 5469 .IP 5470 .nf 5471 \f[C] 5472 rclone genautocomplete zsh [output_file] [flags] 5473 \f[R] 5474 .fi 5475 .SH Options 5476 .IP 5477 .nf 5478 \f[C] 5479 -h, --help help for zsh 5480 \f[R] 5481 .fi 5482 .PP 5483 See the global flags page (https://rclone.org/flags/) for global options 5484 not listed here. 5485 .SH SEE ALSO 5486 .IP \[bu] 2 5487 rclone 5488 genautocomplete (https://rclone.org/commands/rclone_genautocomplete/) - 5489 Output completion script for a given shell. 5490 .SH rclone gendocs 5491 .PP 5492 Output markdown docs for rclone to the directory supplied. 5493 .SS Synopsis 5494 .PP 5495 This produces markdown docs for the rclone commands to the directory 5496 supplied. 5497 These are in a format suitable for hugo to render into the rclone.org 5498 website. 5499 .IP 5500 .nf 5501 \f[C] 5502 rclone gendocs output_directory [flags] 5503 \f[R] 5504 .fi 5505 .SS Options 5506 .IP 5507 .nf 5508 \f[C] 5509 -h, --help help for gendocs 5510 \f[R] 5511 .fi 5512 .PP 5513 See the global flags page (https://rclone.org/flags/) for global options 5514 not listed here. 5515 .SH SEE ALSO 5516 .IP \[bu] 2 5517 rclone (https://rclone.org/commands/rclone/) - Show help for rclone 5518 commands, flags and backends. 5519 .SH rclone hashsum 5520 .PP 5521 Produces a hashsum file for all the objects in the path. 5522 .SS Synopsis 5523 .PP 5524 Produces a hash file for all the objects in the path using the hash 5525 named. 5526 The output is in the same format as the standard md5sum/sha1sum tool. 5527 .PP 5528 By default, the hash is requested from the remote. 5529 If the hash is not supported by the remote, no hash will be returned. 5530 With the download flag, the file will be downloaded from the remote and 5531 hashed locally enabling any hash for any remote. 5532 .PP 5533 For the MD5 and SHA1 algorithms there are also dedicated commands, 5534 md5sum (https://rclone.org/commands/rclone_md5sum/) and 5535 sha1sum (https://rclone.org/commands/rclone_sha1sum/). 5536 .PP 5537 This command can also hash data received on standard input (stdin), by 5538 not passing a remote:path, or by passing a hyphen as remote:path when 5539 there is data to read (if not, the hyphen will be treated literally, as 5540 a relative path). 5541 .PP 5542 Run without a hash to see the list of all supported hashes, e.g. 5543 .IP 5544 .nf 5545 \f[C] 5546 $ rclone hashsum 5547 Supported hashes are: 5548 * md5 5549 * sha1 5550 * whirlpool 5551 * crc32 5552 * sha256 5553 \f[R] 5554 .fi 5555 .PP 5556 Then 5557 .IP 5558 .nf 5559 \f[C] 5560 $ rclone hashsum MD5 remote:path 5561 \f[R] 5562 .fi 5563 .PP 5564 Note that hash names are case insensitive and values are output in lower 5565 case. 5566 .IP 5567 .nf 5568 \f[C] 5569 rclone hashsum [<hash> remote:path] [flags] 5570 \f[R] 5571 .fi 5572 .SS Options 5573 .IP 5574 .nf 5575 \f[C] 5576 --base64 Output base64 encoded hashsum 5577 -C, --checkfile string Validate hashes against a given SUM file instead of printing them 5578 --download Download the file and hash it locally; if this flag is not specified, the hash is requested from the remote 5579 -h, --help help for hashsum 5580 --output-file string Output hashsums to a file rather than the terminal 5581 \f[R] 5582 .fi 5583 .SS Filter Options 5584 .PP 5585 Flags for filtering directory listings. 5586 .IP 5587 .nf 5588 \f[C] 5589 --delete-excluded Delete files on dest excluded from sync 5590 --exclude stringArray Exclude files matching pattern 5591 --exclude-from stringArray Read file exclude patterns from file (use - to read from stdin) 5592 --exclude-if-present stringArray Exclude directories if filename is present 5593 --files-from stringArray Read list of source-file names from file (use - to read from stdin) 5594 --files-from-raw stringArray Read list of source-file names from file without any processing of lines (use - to read from stdin) 5595 -f, --filter stringArray Add a file filtering rule 5596 --filter-from stringArray Read file filtering patterns from a file (use - to read from stdin) 5597 --ignore-case Ignore case in filters (case insensitive) 5598 --include stringArray Include files matching pattern 5599 --include-from stringArray Read file include patterns from file (use - to read from stdin) 5600 --max-age Duration Only transfer files younger than this in s or suffix ms|s|m|h|d|w|M|y (default off) 5601 --max-depth int If set limits the recursion depth to this (default -1) 5602 --max-size SizeSuffix Only transfer files smaller than this in KiB or suffix B|K|M|G|T|P (default off) 5603 --metadata-exclude stringArray Exclude metadatas matching pattern 5604 --metadata-exclude-from stringArray Read metadata exclude patterns from file (use - to read from stdin) 5605 --metadata-filter stringArray Add a metadata filtering rule 5606 --metadata-filter-from stringArray Read metadata filtering patterns from a file (use - to read from stdin) 5607 --metadata-include stringArray Include metadatas matching pattern 5608 --metadata-include-from stringArray Read metadata include patterns from file (use - to read from stdin) 5609 --min-age Duration Only transfer files older than this in s or suffix ms|s|m|h|d|w|M|y (default off) 5610 --min-size SizeSuffix Only transfer files bigger than this in KiB or suffix B|K|M|G|T|P (default off) 5611 \f[R] 5612 .fi 5613 .SS Listing Options 5614 .PP 5615 Flags for listing directories. 5616 .IP 5617 .nf 5618 \f[C] 5619 --default-time Time Time to show if modtime is unknown for files and directories (default 2000-01-01T00:00:00Z) 5620 --fast-list Use recursive list if available; uses more memory but fewer transactions 5621 \f[R] 5622 .fi 5623 .PP 5624 See the global flags page (https://rclone.org/flags/) for global options 5625 not listed here. 5626 .SH SEE ALSO 5627 .IP \[bu] 2 5628 rclone (https://rclone.org/commands/rclone/) - Show help for rclone 5629 commands, flags and backends. 5630 .SH rclone link 5631 .PP 5632 Generate public link to file/folder. 5633 .SS Synopsis 5634 .PP 5635 rclone link will create, retrieve or remove a public link to the given 5636 file or folder. 5637 .IP 5638 .nf 5639 \f[C] 5640 rclone link remote:path/to/file 5641 rclone link remote:path/to/folder/ 5642 rclone link --unlink remote:path/to/folder/ 5643 rclone link --expire 1d remote:path/to/file 5644 \f[R] 5645 .fi 5646 .PP 5647 If you supply the --expire flag, it will set the expiration time 5648 otherwise it will use the default (100 years). 5649 \f[B]Note\f[R] not all backends support the --expire flag - if the 5650 backend doesn\[aq]t support it then the link returned won\[aq]t expire. 5651 .PP 5652 Use the --unlink flag to remove existing public links to the file or 5653 folder. 5654 \f[B]Note\f[R] not all backends support \[dq]--unlink\[dq] flag - those 5655 that don\[aq]t will just ignore it. 5656 .PP 5657 If successful, the last line of the output will contain the link. 5658 Exact capabilities depend on the remote, but the link will always by 5659 default be created with the least constraints \[en] e.g. 5660 no expiry, no password protection, accessible without account. 5661 .IP 5662 .nf 5663 \f[C] 5664 rclone link remote:path [flags] 5665 \f[R] 5666 .fi 5667 .SS Options 5668 .IP 5669 .nf 5670 \f[C] 5671 --expire Duration The amount of time that the link will be valid (default off) 5672 -h, --help help for link 5673 --unlink Remove existing public link to file/folder 5674 \f[R] 5675 .fi 5676 .PP 5677 See the global flags page (https://rclone.org/flags/) for global options 5678 not listed here. 5679 .SH SEE ALSO 5680 .IP \[bu] 2 5681 rclone (https://rclone.org/commands/rclone/) - Show help for rclone 5682 commands, flags and backends. 5683 .SH rclone listremotes 5684 .PP 5685 List all the remotes in the config file and defined in environment 5686 variables. 5687 .SS Synopsis 5688 .PP 5689 rclone listremotes lists all the available remotes from the config file. 5690 .PP 5691 When used with the \f[C]--long\f[R] flag it lists the types and the 5692 descriptions too. 5693 .IP 5694 .nf 5695 \f[C] 5696 rclone listremotes [flags] 5697 \f[R] 5698 .fi 5699 .SS Options 5700 .IP 5701 .nf 5702 \f[C] 5703 -h, --help help for listremotes 5704 --long Show the type and the description as well as names 5705 \f[R] 5706 .fi 5707 .PP 5708 See the global flags page (https://rclone.org/flags/) for global options 5709 not listed here. 5710 .SH SEE ALSO 5711 .IP \[bu] 2 5712 rclone (https://rclone.org/commands/rclone/) - Show help for rclone 5713 commands, flags and backends. 5714 .SH rclone lsf 5715 .PP 5716 List directories and objects in remote:path formatted for parsing. 5717 .SS Synopsis 5718 .PP 5719 List the contents of the source path (directories and objects) to 5720 standard output in a form which is easy to parse by scripts. 5721 By default this will just be the names of the objects and directories, 5722 one per line. 5723 The directories will have a / suffix. 5724 .PP 5725 Eg 5726 .IP 5727 .nf 5728 \f[C] 5729 $ rclone lsf swift:bucket 5730 bevajer5jef 5731 canole 5732 diwogej7 5733 ferejej3gux/ 5734 fubuwic 5735 \f[R] 5736 .fi 5737 .PP 5738 Use the \f[C]--format\f[R] option to control what gets listed. 5739 By default this is just the path, but you can use these parameters to 5740 control the output: 5741 .IP 5742 .nf 5743 \f[C] 5744 p - path 5745 s - size 5746 t - modification time 5747 h - hash 5748 i - ID of object 5749 o - Original ID of underlying object 5750 m - MimeType of object if known 5751 e - encrypted name 5752 T - tier of storage if known, e.g. \[dq]Hot\[dq] or \[dq]Cool\[dq] 5753 M - Metadata of object in JSON blob format, eg {\[dq]key\[dq]:\[dq]value\[dq]} 5754 \f[R] 5755 .fi 5756 .PP 5757 So if you wanted the path, size and modification time, you would use 5758 \f[C]--format \[dq]pst\[dq]\f[R], or maybe 5759 \f[C]--format \[dq]tsp\[dq]\f[R] to put the path last. 5760 .PP 5761 Eg 5762 .IP 5763 .nf 5764 \f[C] 5765 $ rclone lsf --format \[dq]tsp\[dq] swift:bucket 5766 2016-06-25 18:55:41;60295;bevajer5jef 5767 2016-06-25 18:55:43;90613;canole 5768 2016-06-25 18:55:43;94467;diwogej7 5769 2018-04-26 08:50:45;0;ferejej3gux/ 5770 2016-06-25 18:55:40;37600;fubuwic 5771 \f[R] 5772 .fi 5773 .PP 5774 If you specify \[dq]h\[dq] in the format you will get the MD5 hash by 5775 default, use the \f[C]--hash\f[R] flag to change which hash you want. 5776 Note that this can be returned as an empty string if it isn\[aq]t 5777 available on the object (and for directories), \[dq]ERROR\[dq] if there 5778 was an error reading it from the object and \[dq]UNSUPPORTED\[dq] if 5779 that object does not support that hash type. 5780 .PP 5781 For example, to emulate the md5sum command you can use 5782 .IP 5783 .nf 5784 \f[C] 5785 rclone lsf -R --hash MD5 --format hp --separator \[dq] \[dq] --files-only . 5786 \f[R] 5787 .fi 5788 .PP 5789 Eg 5790 .IP 5791 .nf 5792 \f[C] 5793 $ rclone lsf -R --hash MD5 --format hp --separator \[dq] \[dq] --files-only swift:bucket 5794 7908e352297f0f530b84a756f188baa3 bevajer5jef 5795 cd65ac234e6fea5925974a51cdd865cc canole 5796 03b5341b4f234b9d984d03ad076bae91 diwogej7 5797 8fd37c3810dd660778137ac3a66cc06d fubuwic 5798 99713e14a4c4ff553acaf1930fad985b gixacuh7ku 5799 \f[R] 5800 .fi 5801 .PP 5802 (Though \[dq]rclone md5sum .\[dq] is an easier way of typing this.) 5803 .PP 5804 By default the separator is \[dq];\[dq] this can be changed with the 5805 \f[C]--separator\f[R] flag. 5806 Note that separators aren\[aq]t escaped in the path so putting it last 5807 is a good strategy. 5808 .PP 5809 Eg 5810 .IP 5811 .nf 5812 \f[C] 5813 $ rclone lsf --separator \[dq],\[dq] --format \[dq]tshp\[dq] swift:bucket 5814 2016-06-25 18:55:41,60295,7908e352297f0f530b84a756f188baa3,bevajer5jef 5815 2016-06-25 18:55:43,90613,cd65ac234e6fea5925974a51cdd865cc,canole 5816 2016-06-25 18:55:43,94467,03b5341b4f234b9d984d03ad076bae91,diwogej7 5817 2018-04-26 08:52:53,0,,ferejej3gux/ 5818 2016-06-25 18:55:40,37600,8fd37c3810dd660778137ac3a66cc06d,fubuwic 5819 \f[R] 5820 .fi 5821 .PP 5822 You can output in CSV standard format. 5823 This will escape things in \[dq] if they contain , 5824 .PP 5825 Eg 5826 .IP 5827 .nf 5828 \f[C] 5829 $ rclone lsf --csv --files-only --format ps remote:path 5830 test.log,22355 5831 test.sh,449 5832 \[dq]this file contains a comma, in the file name.txt\[dq],6 5833 \f[R] 5834 .fi 5835 .PP 5836 Note that the \f[C]--absolute\f[R] parameter is useful for making lists 5837 of files to pass to an rclone copy with the \f[C]--files-from-raw\f[R] 5838 flag. 5839 .PP 5840 For example, to find all the files modified within one day and copy 5841 those only (without traversing the whole directory structure): 5842 .IP 5843 .nf 5844 \f[C] 5845 rclone lsf --absolute --files-only --max-age 1d /path/to/local > new_files 5846 rclone copy --files-from-raw new_files /path/to/local remote:path 5847 \f[R] 5848 .fi 5849 .PP 5850 The default time format is \f[C]\[aq]2006-01-02 15:04:05\[aq]\f[R]. 5851 Other formats (https://pkg.go.dev/time#pkg-constants) can be specified 5852 with the \f[C]--time-format\f[R] flag. 5853 Examples: 5854 .IP 5855 .nf 5856 \f[C] 5857 rclone lsf remote:path --format pt --time-format \[aq]Jan 2, 2006 at 3:04pm (MST)\[aq] 5858 rclone lsf remote:path --format pt --time-format \[aq]2006-01-02 15:04:05.000000000\[aq] 5859 rclone lsf remote:path --format pt --time-format \[aq]2006-01-02T15:04:05.999999999Z07:00\[aq] 5860 rclone lsf remote:path --format pt --time-format RFC3339 5861 rclone lsf remote:path --format pt --time-format DateOnly 5862 rclone lsf remote:path --format pt --time-format max 5863 \f[R] 5864 .fi 5865 .PP 5866 \f[C]--time-format max\f[R] will automatically truncate 5867 \[aq]\f[C]2006-01-02 15:04:05.000000000\f[R]\[aq] to the maximum 5868 precision supported by the remote. 5869 .PP 5870 Any of the filtering options can be applied to this command. 5871 .PP 5872 There are several related list commands 5873 .IP \[bu] 2 5874 \f[C]ls\f[R] to list size and path of objects only 5875 .IP \[bu] 2 5876 \f[C]lsl\f[R] to list modification time, size and path of objects only 5877 .IP \[bu] 2 5878 \f[C]lsd\f[R] to list directories only 5879 .IP \[bu] 2 5880 \f[C]lsf\f[R] to list objects and directories in easy to parse format 5881 .IP \[bu] 2 5882 \f[C]lsjson\f[R] to list objects and directories in JSON format 5883 .PP 5884 \f[C]ls\f[R],\f[C]lsl\f[R],\f[C]lsd\f[R] are designed to be 5885 human-readable. 5886 \f[C]lsf\f[R] is designed to be human and machine-readable. 5887 \f[C]lsjson\f[R] is designed to be machine-readable. 5888 .PP 5889 Note that \f[C]ls\f[R] and \f[C]lsl\f[R] recurse by default - use 5890 \f[C]--max-depth 1\f[R] to stop the recursion. 5891 .PP 5892 The other list commands \f[C]lsd\f[R],\f[C]lsf\f[R],\f[C]lsjson\f[R] do 5893 not recurse by default - use \f[C]-R\f[R] to make them recurse. 5894 .PP 5895 Listing a nonexistent directory will produce an error except for remotes 5896 which can\[aq]t have empty directories (e.g. 5897 s3, swift, or gcs - the bucket-based remotes). 5898 .IP 5899 .nf 5900 \f[C] 5901 rclone lsf remote:path [flags] 5902 \f[R] 5903 .fi 5904 .SS Options 5905 .IP 5906 .nf 5907 \f[C] 5908 --absolute Put a leading / in front of path names 5909 --csv Output in CSV format 5910 -d, --dir-slash Append a slash to directory names (default true) 5911 --dirs-only Only list directories 5912 --files-only Only list files 5913 -F, --format string Output format - see help for details (default \[dq]p\[dq]) 5914 --hash h Use this hash when h is used in the format MD5|SHA-1|DropboxHash (default \[dq]md5\[dq]) 5915 -h, --help help for lsf 5916 -R, --recursive Recurse into the listing 5917 -s, --separator string Separator for the items in the format (default \[dq];\[dq]) 5918 -t, --time-format string Specify a custom time format, or \[aq]max\[aq] for max precision supported by remote (default: 2006-01-02 15:04:05) 5919 \f[R] 5920 .fi 5921 .SS Filter Options 5922 .PP 5923 Flags for filtering directory listings. 5924 .IP 5925 .nf 5926 \f[C] 5927 --delete-excluded Delete files on dest excluded from sync 5928 --exclude stringArray Exclude files matching pattern 5929 --exclude-from stringArray Read file exclude patterns from file (use - to read from stdin) 5930 --exclude-if-present stringArray Exclude directories if filename is present 5931 --files-from stringArray Read list of source-file names from file (use - to read from stdin) 5932 --files-from-raw stringArray Read list of source-file names from file without any processing of lines (use - to read from stdin) 5933 -f, --filter stringArray Add a file filtering rule 5934 --filter-from stringArray Read file filtering patterns from a file (use - to read from stdin) 5935 --ignore-case Ignore case in filters (case insensitive) 5936 --include stringArray Include files matching pattern 5937 --include-from stringArray Read file include patterns from file (use - to read from stdin) 5938 --max-age Duration Only transfer files younger than this in s or suffix ms|s|m|h|d|w|M|y (default off) 5939 --max-depth int If set limits the recursion depth to this (default -1) 5940 --max-size SizeSuffix Only transfer files smaller than this in KiB or suffix B|K|M|G|T|P (default off) 5941 --metadata-exclude stringArray Exclude metadatas matching pattern 5942 --metadata-exclude-from stringArray Read metadata exclude patterns from file (use - to read from stdin) 5943 --metadata-filter stringArray Add a metadata filtering rule 5944 --metadata-filter-from stringArray Read metadata filtering patterns from a file (use - to read from stdin) 5945 --metadata-include stringArray Include metadatas matching pattern 5946 --metadata-include-from stringArray Read metadata include patterns from file (use - to read from stdin) 5947 --min-age Duration Only transfer files older than this in s or suffix ms|s|m|h|d|w|M|y (default off) 5948 --min-size SizeSuffix Only transfer files bigger than this in KiB or suffix B|K|M|G|T|P (default off) 5949 \f[R] 5950 .fi 5951 .SS Listing Options 5952 .PP 5953 Flags for listing directories. 5954 .IP 5955 .nf 5956 \f[C] 5957 --default-time Time Time to show if modtime is unknown for files and directories (default 2000-01-01T00:00:00Z) 5958 --fast-list Use recursive list if available; uses more memory but fewer transactions 5959 \f[R] 5960 .fi 5961 .PP 5962 See the global flags page (https://rclone.org/flags/) for global options 5963 not listed here. 5964 .SH SEE ALSO 5965 .IP \[bu] 2 5966 rclone (https://rclone.org/commands/rclone/) - Show help for rclone 5967 commands, flags and backends. 5968 .SH rclone lsjson 5969 .PP 5970 List directories and objects in the path in JSON format. 5971 .SS Synopsis 5972 .PP 5973 List directories and objects in the path in JSON format. 5974 .PP 5975 The output is an array of Items, where each Item looks like this 5976 .IP 5977 .nf 5978 \f[C] 5979 { 5980 \[dq]Hashes\[dq] : { 5981 \[dq]SHA-1\[dq] : \[dq]f572d396fae9206628714fb2ce00f72e94f2258f\[dq], 5982 \[dq]MD5\[dq] : \[dq]b1946ac92492d2347c6235b4d2611184\[dq], 5983 \[dq]DropboxHash\[dq] : \[dq]ecb65bb98f9d905b70458986c39fcbad7715e5f2fcc3b1f07767d7c83e2438cc\[dq] 5984 }, 5985 \[dq]ID\[dq]: \[dq]y2djkhiujf83u33\[dq], 5986 \[dq]OrigID\[dq]: \[dq]UYOJVTUW00Q1RzTDA\[dq], 5987 \[dq]IsBucket\[dq] : false, 5988 \[dq]IsDir\[dq] : false, 5989 \[dq]MimeType\[dq] : \[dq]application/octet-stream\[dq], 5990 \[dq]ModTime\[dq] : \[dq]2017-05-31T16:15:57.034468261+01:00\[dq], 5991 \[dq]Name\[dq] : \[dq]file.txt\[dq], 5992 \[dq]Encrypted\[dq] : \[dq]v0qpsdq8anpci8n929v3uu9338\[dq], 5993 \[dq]EncryptedPath\[dq] : \[dq]kja9098349023498/v0qpsdq8anpci8n929v3uu9338\[dq], 5994 \[dq]Path\[dq] : \[dq]full/path/goes/here/file.txt\[dq], 5995 \[dq]Size\[dq] : 6, 5996 \[dq]Tier\[dq] : \[dq]hot\[dq], 5997 } 5998 \f[R] 5999 .fi 6000 .PP 6001 If \f[C]--hash\f[R] is not specified the Hashes property won\[aq]t be 6002 emitted. 6003 The types of hash can be specified with the \f[C]--hash-type\f[R] 6004 parameter (which may be repeated). 6005 If \f[C]--hash-type\f[R] is set then it implies \f[C]--hash\f[R]. 6006 .PP 6007 If \f[C]--no-modtime\f[R] is specified then ModTime will be blank. 6008 This can speed things up on remotes where reading the ModTime takes an 6009 extra request (e.g. 6010 s3, swift). 6011 .PP 6012 If \f[C]--no-mimetype\f[R] is specified then MimeType will be blank. 6013 This can speed things up on remotes where reading the MimeType takes an 6014 extra request (e.g. 6015 s3, swift). 6016 .PP 6017 If \f[C]--encrypted\f[R] is not specified the Encrypted won\[aq]t be 6018 emitted. 6019 .PP 6020 If \f[C]--dirs-only\f[R] is not specified files in addition to 6021 directories are returned 6022 .PP 6023 If \f[C]--files-only\f[R] is not specified directories in addition to 6024 the files will be returned. 6025 .PP 6026 If \f[C]--metadata\f[R] is set then an additional Metadata key will be 6027 returned. 6028 This will have metadata in rclone standard format as a JSON object. 6029 .PP 6030 if \f[C]--stat\f[R] is set then a single JSON blob will be returned 6031 about the item pointed to. 6032 This will return an error if the item isn\[aq]t found. 6033 However on bucket based backends (like s3, gcs, b2, azureblob etc) if 6034 the item isn\[aq]t found it will return an empty directory as it 6035 isn\[aq]t possible to tell empty directories from missing directories 6036 there. 6037 .PP 6038 The Path field will only show folders below the remote path being 6039 listed. 6040 If \[dq]remote:path\[dq] contains the file \[dq]subfolder/file.txt\[dq], 6041 the Path for \[dq]file.txt\[dq] will be \[dq]subfolder/file.txt\[dq], 6042 not \[dq]remote:path/subfolder/file.txt\[dq]. 6043 When used without \f[C]--recursive\f[R] the Path will always be the same 6044 as Name. 6045 .PP 6046 If the directory is a bucket in a bucket-based backend, then 6047 \[dq]IsBucket\[dq] will be set to true. 6048 This key won\[aq]t be present unless it is \[dq]true\[dq]. 6049 .PP 6050 The time is in RFC3339 format with up to nanosecond precision. 6051 The number of decimal digits in the seconds will depend on the precision 6052 that the remote can hold the times, so if times are accurate to the 6053 nearest millisecond (e.g. 6054 Google Drive) then 3 digits will always be shown 6055 (\[dq]2017-05-31T16:15:57.034+01:00\[dq]) whereas if the times are 6056 accurate to the nearest second (Dropbox, Box, WebDav, etc.) no digits 6057 will be shown (\[dq]2017-05-31T16:15:57+01:00\[dq]). 6058 .PP 6059 The whole output can be processed as a JSON blob, or alternatively it 6060 can be processed line by line as each item is written one to a line. 6061 .PP 6062 Any of the filtering options can be applied to this command. 6063 .PP 6064 There are several related list commands 6065 .IP \[bu] 2 6066 \f[C]ls\f[R] to list size and path of objects only 6067 .IP \[bu] 2 6068 \f[C]lsl\f[R] to list modification time, size and path of objects only 6069 .IP \[bu] 2 6070 \f[C]lsd\f[R] to list directories only 6071 .IP \[bu] 2 6072 \f[C]lsf\f[R] to list objects and directories in easy to parse format 6073 .IP \[bu] 2 6074 \f[C]lsjson\f[R] to list objects and directories in JSON format 6075 .PP 6076 \f[C]ls\f[R],\f[C]lsl\f[R],\f[C]lsd\f[R] are designed to be 6077 human-readable. 6078 \f[C]lsf\f[R] is designed to be human and machine-readable. 6079 \f[C]lsjson\f[R] is designed to be machine-readable. 6080 .PP 6081 Note that \f[C]ls\f[R] and \f[C]lsl\f[R] recurse by default - use 6082 \f[C]--max-depth 1\f[R] to stop the recursion. 6083 .PP 6084 The other list commands \f[C]lsd\f[R],\f[C]lsf\f[R],\f[C]lsjson\f[R] do 6085 not recurse by default - use \f[C]-R\f[R] to make them recurse. 6086 .PP 6087 Listing a nonexistent directory will produce an error except for remotes 6088 which can\[aq]t have empty directories (e.g. 6089 s3, swift, or gcs - the bucket-based remotes). 6090 .IP 6091 .nf 6092 \f[C] 6093 rclone lsjson remote:path [flags] 6094 \f[R] 6095 .fi 6096 .SS Options 6097 .IP 6098 .nf 6099 \f[C] 6100 --dirs-only Show only directories in the listing 6101 --encrypted Show the encrypted names 6102 --files-only Show only files in the listing 6103 --hash Include hashes in the output (may take longer) 6104 --hash-type stringArray Show only this hash type (may be repeated) 6105 -h, --help help for lsjson 6106 -M, --metadata Add metadata to the listing 6107 --no-mimetype Don\[aq]t read the mime type (can speed things up) 6108 --no-modtime Don\[aq]t read the modification time (can speed things up) 6109 --original Show the ID of the underlying Object 6110 -R, --recursive Recurse into the listing 6111 --stat Just return the info for the pointed to file 6112 \f[R] 6113 .fi 6114 .SS Filter Options 6115 .PP 6116 Flags for filtering directory listings. 6117 .IP 6118 .nf 6119 \f[C] 6120 --delete-excluded Delete files on dest excluded from sync 6121 --exclude stringArray Exclude files matching pattern 6122 --exclude-from stringArray Read file exclude patterns from file (use - to read from stdin) 6123 --exclude-if-present stringArray Exclude directories if filename is present 6124 --files-from stringArray Read list of source-file names from file (use - to read from stdin) 6125 --files-from-raw stringArray Read list of source-file names from file without any processing of lines (use - to read from stdin) 6126 -f, --filter stringArray Add a file filtering rule 6127 --filter-from stringArray Read file filtering patterns from a file (use - to read from stdin) 6128 --ignore-case Ignore case in filters (case insensitive) 6129 --include stringArray Include files matching pattern 6130 --include-from stringArray Read file include patterns from file (use - to read from stdin) 6131 --max-age Duration Only transfer files younger than this in s or suffix ms|s|m|h|d|w|M|y (default off) 6132 --max-depth int If set limits the recursion depth to this (default -1) 6133 --max-size SizeSuffix Only transfer files smaller than this in KiB or suffix B|K|M|G|T|P (default off) 6134 --metadata-exclude stringArray Exclude metadatas matching pattern 6135 --metadata-exclude-from stringArray Read metadata exclude patterns from file (use - to read from stdin) 6136 --metadata-filter stringArray Add a metadata filtering rule 6137 --metadata-filter-from stringArray Read metadata filtering patterns from a file (use - to read from stdin) 6138 --metadata-include stringArray Include metadatas matching pattern 6139 --metadata-include-from stringArray Read metadata include patterns from file (use - to read from stdin) 6140 --min-age Duration Only transfer files older than this in s or suffix ms|s|m|h|d|w|M|y (default off) 6141 --min-size SizeSuffix Only transfer files bigger than this in KiB or suffix B|K|M|G|T|P (default off) 6142 \f[R] 6143 .fi 6144 .SS Listing Options 6145 .PP 6146 Flags for listing directories. 6147 .IP 6148 .nf 6149 \f[C] 6150 --default-time Time Time to show if modtime is unknown for files and directories (default 2000-01-01T00:00:00Z) 6151 --fast-list Use recursive list if available; uses more memory but fewer transactions 6152 \f[R] 6153 .fi 6154 .PP 6155 See the global flags page (https://rclone.org/flags/) for global options 6156 not listed here. 6157 .SH SEE ALSO 6158 .IP \[bu] 2 6159 rclone (https://rclone.org/commands/rclone/) - Show help for rclone 6160 commands, flags and backends. 6161 .SH rclone mount 6162 .PP 6163 Mount the remote as file system on a mountpoint. 6164 .SS Synopsis 6165 .PP 6166 rclone mount allows Linux, FreeBSD, macOS and Windows to mount any of 6167 Rclone\[aq]s cloud storage systems as a file system with FUSE. 6168 .PP 6169 First set up your remote using \f[C]rclone config\f[R]. 6170 Check it works with \f[C]rclone ls\f[R] etc. 6171 .PP 6172 On Linux and macOS, you can run mount in either foreground or background 6173 (aka daemon) mode. 6174 Mount runs in foreground mode by default. 6175 Use the \f[C]--daemon\f[R] flag to force background mode. 6176 On Windows you can run mount in foreground only, the flag is ignored. 6177 .PP 6178 In background mode rclone acts as a generic Unix mount program: the main 6179 program starts, spawns background rclone process to setup and maintain 6180 the mount, waits until success or timeout and exits with appropriate 6181 code (killing the child process if it fails). 6182 .PP 6183 On Linux/macOS/FreeBSD start the mount like this, where 6184 \f[C]/path/to/local/mount\f[R] is an \f[B]empty\f[R] \f[B]existing\f[R] 6185 directory: 6186 .IP 6187 .nf 6188 \f[C] 6189 rclone mount remote:path/to/files /path/to/local/mount 6190 \f[R] 6191 .fi 6192 .PP 6193 On Windows you can start a mount in different ways. 6194 See below for details. 6195 If foreground mount is used interactively from a console window, rclone 6196 will serve the mount and occupy the console so another window should be 6197 used to work with the mount until rclone is interrupted e.g. 6198 by pressing Ctrl-C. 6199 .PP 6200 The following examples will mount to an automatically assigned drive, to 6201 specific drive letter \f[C]X:\f[R], to path 6202 \f[C]C:\[rs]path\[rs]parent\[rs]mount\f[R] (where parent directory or 6203 drive must exist, and mount must \f[B]not\f[R] exist, and is not 6204 supported when mounting as a network drive), and the last example will 6205 mount as network share \f[C]\[rs]\[rs]cloud\[rs]remote\f[R] and map it 6206 to an automatically assigned drive: 6207 .IP 6208 .nf 6209 \f[C] 6210 rclone mount remote:path/to/files * 6211 rclone mount remote:path/to/files X: 6212 rclone mount remote:path/to/files C:\[rs]path\[rs]parent\[rs]mount 6213 rclone mount remote:path/to/files \[rs]\[rs]cloud\[rs]remote 6214 \f[R] 6215 .fi 6216 .PP 6217 When the program ends while in foreground mode, either via Ctrl+C or 6218 receiving a SIGINT or SIGTERM signal, the mount should be automatically 6219 stopped. 6220 .PP 6221 When running in background mode the user will have to stop the mount 6222 manually: 6223 .IP 6224 .nf 6225 \f[C] 6226 # Linux 6227 fusermount -u /path/to/local/mount 6228 # OS X 6229 umount /path/to/local/mount 6230 \f[R] 6231 .fi 6232 .PP 6233 The umount operation can fail, for example when the mountpoint is busy. 6234 When that happens, it is the user\[aq]s responsibility to stop the mount 6235 manually. 6236 .PP 6237 The size of the mounted file system will be set according to information 6238 retrieved from the remote, the same as returned by the rclone 6239 about (https://rclone.org/commands/rclone_about/) command. 6240 Remotes with unlimited storage may report the used size only, then an 6241 additional 1 PiB of free space is assumed. 6242 If the remote does not 6243 support (https://rclone.org/overview/#optional-features) the about 6244 feature at all, then 1 PiB is set as both the total and the free size. 6245 .SS Installing on Windows 6246 .PP 6247 To run rclone mount on Windows, you will need to download and install 6248 WinFsp (http://www.secfs.net/winfsp/). 6249 .PP 6250 WinFsp (https://github.com/winfsp/winfsp) is an open-source Windows File 6251 System Proxy which makes it easy to write user space file systems for 6252 Windows. 6253 It provides a FUSE emulation layer which rclone uses combination with 6254 cgofuse (https://github.com/winfsp/cgofuse). 6255 Both of these packages are by Bill Zissimopoulos who was very helpful 6256 during the implementation of rclone mount for Windows. 6257 .SS Mounting modes on windows 6258 .PP 6259 Unlike other operating systems, Microsoft Windows provides a different 6260 filesystem type for network and fixed drives. 6261 It optimises access on the assumption fixed disk drives are fast and 6262 reliable, while network drives have relatively high latency and less 6263 reliability. 6264 Some settings can also be differentiated between the two types, for 6265 example that Windows Explorer should just display icons and not create 6266 preview thumbnails for image and video files on network drives. 6267 .PP 6268 In most cases, rclone will mount the remote as a normal, fixed disk 6269 drive by default. 6270 However, you can also choose to mount it as a remote network drive, 6271 often described as a network share. 6272 If you mount an rclone remote using the default, fixed drive mode and 6273 experience unexpected program errors, freezes or other issues, consider 6274 mounting as a network drive instead. 6275 .PP 6276 When mounting as a fixed disk drive you can either mount to an unused 6277 drive letter, or to a path representing a \f[B]nonexistent\f[R] 6278 subdirectory of an \f[B]existing\f[R] parent directory or drive. 6279 Using the special value \f[C]*\f[R] will tell rclone to automatically 6280 assign the next available drive letter, starting with Z: and moving 6281 backward. 6282 Examples: 6283 .IP 6284 .nf 6285 \f[C] 6286 rclone mount remote:path/to/files * 6287 rclone mount remote:path/to/files X: 6288 rclone mount remote:path/to/files C:\[rs]path\[rs]parent\[rs]mount 6289 rclone mount remote:path/to/files X: 6290 \f[R] 6291 .fi 6292 .PP 6293 Option \f[C]--volname\f[R] can be used to set a custom volume name for 6294 the mounted file system. 6295 The default is to use the remote name and path. 6296 .PP 6297 To mount as network drive, you can add option \f[C]--network-mode\f[R] 6298 to your mount command. 6299 Mounting to a directory path is not supported in this mode, it is a 6300 limitation Windows imposes on junctions, so the remote must always be 6301 mounted to a drive letter. 6302 .IP 6303 .nf 6304 \f[C] 6305 rclone mount remote:path/to/files X: --network-mode 6306 \f[R] 6307 .fi 6308 .PP 6309 A volume name specified with \f[C]--volname\f[R] will be used to create 6310 the network share path. 6311 A complete UNC path, such as \f[C]\[rs]\[rs]cloud\[rs]remote\f[R], 6312 optionally with path 6313 \f[C]\[rs]\[rs]cloud\[rs]remote\[rs]madeup\[rs]path\f[R], will be used 6314 as is. 6315 Any other string will be used as the share part, after a default prefix 6316 \f[C]\[rs]\[rs]server\[rs]\f[R]. 6317 If no volume name is specified then \f[C]\[rs]\[rs]server\[rs]share\f[R] 6318 will be used. 6319 You must make sure the volume name is unique when you are mounting more 6320 than one drive, or else the mount command will fail. 6321 The share name will treated as the volume label for the mapped drive, 6322 shown in Windows Explorer etc, while the complete 6323 \f[C]\[rs]\[rs]server\[rs]share\f[R] will be reported as the remote UNC 6324 path by \f[C]net use\f[R] etc, just like a normal network drive mapping. 6325 .PP 6326 If you specify a full network share UNC path with \f[C]--volname\f[R], 6327 this will implicitly set the \f[C]--network-mode\f[R] option, so the 6328 following two examples have same result: 6329 .IP 6330 .nf 6331 \f[C] 6332 rclone mount remote:path/to/files X: --network-mode 6333 rclone mount remote:path/to/files X: --volname \[rs]\[rs]server\[rs]share 6334 \f[R] 6335 .fi 6336 .PP 6337 You may also specify the network share UNC path as the mountpoint 6338 itself. 6339 Then rclone will automatically assign a drive letter, same as with 6340 \f[C]*\f[R] and use that as mountpoint, and instead use the UNC path 6341 specified as the volume name, as if it were specified with the 6342 \f[C]--volname\f[R] option. 6343 This will also implicitly set the \f[C]--network-mode\f[R] option. 6344 This means the following two examples have same result: 6345 .IP 6346 .nf 6347 \f[C] 6348 rclone mount remote:path/to/files \[rs]\[rs]cloud\[rs]remote 6349 rclone mount remote:path/to/files * --volname \[rs]\[rs]cloud\[rs]remote 6350 \f[R] 6351 .fi 6352 .PP 6353 There is yet another way to enable network mode, and to set the share 6354 path, and that is to pass the \[dq]native\[dq] libfuse/WinFsp option 6355 directly: \f[C]--fuse-flag --VolumePrefix=\[rs]server\[rs]share\f[R]. 6356 Note that the path must be with just a single backslash prefix in this 6357 case. 6358 .PP 6359 \f[I]Note:\f[R] In previous versions of rclone this was the only 6360 supported method. 6361 .PP 6362 Read more about drive 6363 mapping (https://en.wikipedia.org/wiki/Drive_mapping) 6364 .PP 6365 See also Limitations section below. 6366 .SS Windows filesystem permissions 6367 .PP 6368 The FUSE emulation layer on Windows must convert between the POSIX-based 6369 permission model used in FUSE, and the permission model used in Windows, 6370 based on access-control lists (ACL). 6371 .PP 6372 The mounted filesystem will normally get three entries in its 6373 access-control list (ACL), representing permissions for the POSIX 6374 permission scopes: Owner, group and others. 6375 By default, the owner and group will be taken from the current user, and 6376 the built-in group \[dq]Everyone\[dq] will be used to represent others. 6377 The user/group can be customized with FUSE options \[dq]UserName\[dq] 6378 and \[dq]GroupName\[dq], e.g. 6379 \f[C]-o UserName=user123 -o GroupName=\[dq]Authenticated Users\[dq]\f[R]. 6380 The permissions on each entry will be set according to options 6381 \f[C]--dir-perms\f[R] and \f[C]--file-perms\f[R], which takes a value in 6382 traditional Unix numeric 6383 notation (https://en.wikipedia.org/wiki/File-system_permissions#Numeric_notation). 6384 .PP 6385 The default permissions corresponds to 6386 \f[C]--file-perms 0666 --dir-perms 0777\f[R], i.e. 6387 read and write permissions to everyone. 6388 This means you will not be able to start any programs from the mount. 6389 To be able to do that you must add execute permissions, e.g. 6390 \f[C]--file-perms 0777 --dir-perms 0777\f[R] to add it to everyone. 6391 If the program needs to write files, chances are you will have to enable 6392 VFS File Caching as well (see also limitations). 6393 Note that the default write permission have some restrictions for 6394 accounts other than the owner, specifically it lacks the \[dq]write 6395 extended attributes\[dq], as explained next. 6396 .PP 6397 The mapping of permissions is not always trivial, and the result you see 6398 in Windows Explorer may not be exactly like you expected. 6399 For example, when setting a value that includes write access for the 6400 group or others scope, this will be mapped to individual permissions 6401 \[dq]write attributes\[dq], \[dq]write data\[dq] and \[dq]append 6402 data\[dq], but not \[dq]write extended attributes\[dq]. 6403 Windows will then show this as basic permission \[dq]Special\[dq] 6404 instead of \[dq]Write\[dq], because \[dq]Write\[dq] also covers the 6405 \[dq]write extended attributes\[dq] permission. 6406 When setting digit 0 for group or others, to indicate no permissions, 6407 they will still get individual permissions \[dq]read attributes\[dq], 6408 \[dq]read extended attributes\[dq] and \[dq]read permissions\[dq]. 6409 This is done for compatibility reasons, e.g. 6410 to allow users without additional permissions to be able to read basic 6411 metadata about files like in Unix. 6412 .PP 6413 WinFsp 2021 (version 1.9) introduced a new FUSE option 6414 \[dq]FileSecurity\[dq], that allows the complete specification of file 6415 security descriptors using 6416 SDDL (https://docs.microsoft.com/en-us/windows/win32/secauthz/security-descriptor-string-format). 6417 With this you get detailed control of the resulting permissions, 6418 compared to use of the POSIX permissions described above, and no 6419 additional permissions will be added automatically for compatibility 6420 with Unix. 6421 Some example use cases will following. 6422 .PP 6423 If you set POSIX permissions for only allowing access to the owner, 6424 using \f[C]--file-perms 0600 --dir-perms 0700\f[R], the user group and 6425 the built-in \[dq]Everyone\[dq] group will still be given some special 6426 permissions, as described above. 6427 Some programs may then (incorrectly) interpret this as the file being 6428 accessible by everyone, for example an SSH client may warn about 6429 \[dq]unprotected private key file\[dq]. 6430 You can work around this by specifying 6431 \f[C]-o FileSecurity=\[dq]D:P(A;;FA;;;OW)\[dq]\f[R], which sets file all 6432 access (FA) to the owner (OW), and nothing else. 6433 .PP 6434 When setting write permissions then, except for the owner, this does not 6435 include the \[dq]write extended attributes\[dq] permission, as mentioned 6436 above. 6437 This may prevent applications from writing to files, giving permission 6438 denied error instead. 6439 To set working write permissions for the built-in \[dq]Everyone\[dq] 6440 group, similar to what it gets by default but with the addition of the 6441 \[dq]write extended attributes\[dq], you can specify 6442 \f[C]-o FileSecurity=\[dq]D:P(A;;FRFW;;;WD)\[dq]\f[R], which sets file 6443 read (FR) and file write (FW) to everyone (WD). 6444 If file execute (FX) is also needed, then change to 6445 \f[C]-o FileSecurity=\[dq]D:P(A;;FRFWFX;;;WD)\[dq]\f[R], or set file all 6446 access (FA) to get full access permissions, including delete, with 6447 \f[C]-o FileSecurity=\[dq]D:P(A;;FA;;;WD)\[dq]\f[R]. 6448 .SS Windows caveats 6449 .PP 6450 Drives created as Administrator are not visible to other accounts, not 6451 even an account that was elevated to Administrator with the User Account 6452 Control (UAC) feature. 6453 A result of this is that if you mount to a drive letter from a Command 6454 Prompt run as Administrator, and then try to access the same drive from 6455 Windows Explorer (which does not run as Administrator), you will not be 6456 able to see the mounted drive. 6457 .PP 6458 If you don\[aq]t need to access the drive from applications running with 6459 administrative privileges, the easiest way around this is to always 6460 create the mount from a non-elevated command prompt. 6461 .PP 6462 To make mapped drives available to the user account that created them 6463 regardless if elevated or not, there is a special Windows setting called 6464 linked 6465 connections (https://docs.microsoft.com/en-us/troubleshoot/windows-client/networking/mapped-drives-not-available-from-elevated-command#detail-to-configure-the-enablelinkedconnections-registry-entry) 6466 that can be enabled. 6467 .PP 6468 It is also possible to make a drive mount available to everyone on the 6469 system, by running the process creating it as the built-in SYSTEM 6470 account. 6471 There are several ways to do this: One is to use the command-line 6472 utility 6473 PsExec (https://docs.microsoft.com/en-us/sysinternals/downloads/psexec), 6474 from Microsoft\[aq]s Sysinternals suite, which has option \f[C]-s\f[R] 6475 to start processes as the SYSTEM account. 6476 Another alternative is to run the mount command from a Windows Scheduled 6477 Task, or a Windows Service, configured to run as the SYSTEM account. 6478 A third alternative is to use the WinFsp.Launcher 6479 infrastructure (https://github.com/winfsp/winfsp/wiki/WinFsp-Service-Architecture)). 6480 Read more in the install documentation (https://rclone.org/install/). 6481 Note that when running rclone as another user, it will not use the 6482 configuration file from your profile unless you tell it to with the 6483 \f[C]--config\f[R] (https://rclone.org/docs/#config-config-file) option. 6484 Note also that it is now the SYSTEM account that will have the owner 6485 permissions, and other accounts will have permissions according to the 6486 group or others scopes. 6487 As mentioned above, these will then not get the \[dq]write extended 6488 attributes\[dq] permission, and this may prevent writing to files. 6489 You can work around this with the FileSecurity option, see example 6490 above. 6491 .PP 6492 Note that mapping to a directory path, instead of a drive letter, does 6493 not suffer from the same limitations. 6494 .SS Mounting on macOS 6495 .PP 6496 Mounting on macOS can be done either via built-in NFS 6497 server (https://rclone.org/commands/rclone_serve_nfs/), 6498 macFUSE (https://osxfuse.github.io/) (also known as osxfuse) or 6499 FUSE-T (https://www.fuse-t.org/). 6500 macFUSE is a traditional FUSE driver utilizing a macOS kernel extension 6501 (kext). 6502 FUSE-T is an alternative FUSE system which \[dq]mounts\[dq] via an NFSv4 6503 local server. 6504 .SS Unicode Normalization 6505 .PP 6506 It is highly recommended to keep the default of 6507 \f[C]--no-unicode-normalization=false\f[R] for all \f[C]mount\f[R] and 6508 \f[C]serve\f[R] commands on macOS. 6509 For details, see 6510 vfs-case-sensitivity (https://rclone.org/commands/rclone_mount/#vfs-case-sensitivity). 6511 .SS NFS mount 6512 .PP 6513 This method spins up an NFS server using serve 6514 nfs (https://rclone.org/commands/rclone_serve_nfs/) command and mounts 6515 it to the specified mountpoint. 6516 If you run this in background mode using |--daemon|, you will need to 6517 send SIGTERM signal to the rclone process using |kill| command to stop 6518 the mount. 6519 .PP 6520 Note that \f[C]--nfs-cache-handle-limit\f[R] controls the maximum number 6521 of cached file handles stored by the \f[C]nfsmount\f[R] caching handler. 6522 This should not be set too low or you may experience errors when trying 6523 to access files. 6524 The default is 1000000, but consider lowering this limit if the 6525 server\[aq]s system resource usage causes problems. 6526 .SS macFUSE Notes 6527 .PP 6528 If installing macFUSE using dmg 6529 packages (https://github.com/osxfuse/osxfuse/releases) from the website, 6530 rclone will locate the macFUSE libraries without any further 6531 intervention. 6532 If however, macFUSE is installed using the 6533 macports (https://www.macports.org/) package manager, the following 6534 addition steps are required. 6535 .IP 6536 .nf 6537 \f[C] 6538 sudo mkdir /usr/local/lib 6539 cd /usr/local/lib 6540 sudo ln -s /opt/local/lib/libfuse.2.dylib 6541 \f[R] 6542 .fi 6543 .SS FUSE-T Limitations, Caveats, and Notes 6544 .PP 6545 There are some limitations, caveats, and notes about how it works. 6546 These are current as of FUSE-T version 1.0.14. 6547 .SS ModTime update on read 6548 .PP 6549 As per the FUSE-T 6550 wiki (https://github.com/macos-fuse-t/fuse-t/wiki#caveats): 6551 .RS 6552 .PP 6553 File access and modification times cannot be set separately as it seems 6554 to be an issue with the NFS client which always modifies both. 6555 Can be reproduced with \[aq]touch -m\[aq] and \[aq]touch -a\[aq] 6556 commands 6557 .RE 6558 .PP 6559 This means that viewing files with various tools, notably macOS Finder, 6560 will cause rlcone to update the modification time of the file. 6561 This may make rclone upload a full new copy of the file. 6562 .SS Read Only mounts 6563 .PP 6564 When mounting with \f[C]--read-only\f[R], attempts to write to files 6565 will fail \f[I]silently\f[R] as opposed to with a clear warning as in 6566 macFUSE. 6567 .SS Limitations 6568 .PP 6569 Without the use of \f[C]--vfs-cache-mode\f[R] this can only write files 6570 sequentially, it can only seek when reading. 6571 This means that many applications won\[aq]t work with their files on an 6572 rclone mount without \f[C]--vfs-cache-mode writes\f[R] or 6573 \f[C]--vfs-cache-mode full\f[R]. 6574 See the VFS File Caching section for more info. 6575 When using NFS mount on macOS, if you don\[aq]t specify 6576 |--vfs-cache-mode| the mount point will be read-only. 6577 .PP 6578 The bucket-based remotes (e.g. 6579 Swift, S3, Google Compute Storage, B2) do not support the concept of 6580 empty directories, so empty directories will have a tendency to 6581 disappear once they fall out of the directory cache. 6582 .PP 6583 When \f[C]rclone mount\f[R] is invoked on Unix with \f[C]--daemon\f[R] 6584 flag, the main rclone program will wait for the background mount to 6585 become ready or until the timeout specified by the 6586 \f[C]--daemon-wait\f[R] flag. 6587 On Linux it can check mount status using ProcFS so the flag in fact sets 6588 \f[B]maximum\f[R] time to wait, while the real wait can be less. 6589 On macOS / BSD the time to wait is constant and the check is performed 6590 only at the end. 6591 We advise you to set wait time on macOS reasonably. 6592 .PP 6593 Only supported on Linux, FreeBSD, OS X and Windows at the moment. 6594 .SS rclone mount vs rclone sync/copy 6595 .PP 6596 File systems expect things to be 100% reliable, whereas cloud storage 6597 systems are a long way from 100% reliable. 6598 The rclone sync/copy commands cope with this with lots of retries. 6599 However rclone mount can\[aq]t use retries in the same way without 6600 making local copies of the uploads. 6601 Look at the VFS File Caching for solutions to make mount more reliable. 6602 .SS Attribute caching 6603 .PP 6604 You can use the flag \f[C]--attr-timeout\f[R] to set the time the kernel 6605 caches the attributes (size, modification time, etc.) for directory 6606 entries. 6607 .PP 6608 The default is \f[C]1s\f[R] which caches files just long enough to avoid 6609 too many callbacks to rclone from the kernel. 6610 .PP 6611 In theory 0s should be the correct value for filesystems which can 6612 change outside the control of the kernel. 6613 However this causes quite a few problems such as rclone using too much 6614 memory (https://github.com/artpar/artpar/issues/2157), rclone not 6615 serving files to 6616 samba (https://forum.rclone.org/t/rclone-1-39-vs-1-40-mount-issue/5112) 6617 and excessive time listing 6618 directories (https://github.com/artpar/artpar/issues/2095#issuecomment-371141147). 6619 .PP 6620 The kernel can cache the info about a file for the time given by 6621 \f[C]--attr-timeout\f[R]. 6622 You may see corruption if the remote file changes length during this 6623 window. 6624 It will show up as either a truncated file or a file with garbage on the 6625 end. 6626 With \f[C]--attr-timeout 1s\f[R] this is very unlikely but not 6627 impossible. 6628 The higher you set \f[C]--attr-timeout\f[R] the more likely it is. 6629 The default setting of \[dq]1s\[dq] is the lowest setting which 6630 mitigates the problems above. 6631 .PP 6632 If you set it higher (\f[C]10s\f[R] or \f[C]1m\f[R] say) then the kernel 6633 will call back to rclone less often making it more efficient, however 6634 there is more chance of the corruption issue above. 6635 .PP 6636 If files don\[aq]t change on the remote outside of the control of rclone 6637 then there is no chance of corruption. 6638 .PP 6639 This is the same as setting the attr_timeout option in mount.fuse. 6640 .SS Filters 6641 .PP 6642 Note that all the rclone filters can be used to select a subset of the 6643 files to be visible in the mount. 6644 .SS systemd 6645 .PP 6646 When running rclone mount as a systemd service, it is possible to use 6647 Type=notify. 6648 In this case the service will enter the started state after the 6649 mountpoint has been successfully set up. 6650 Units having the rclone mount service specified as a requirement will 6651 see all files and folders immediately in this mode. 6652 .PP 6653 Note that systemd runs mount units without any environment variables 6654 including \f[C]PATH\f[R] or \f[C]HOME\f[R]. 6655 This means that tilde (\f[C]\[ti]\f[R]) expansion will not work and you 6656 should provide \f[C]--config\f[R] and \f[C]--cache-dir\f[R] explicitly 6657 as absolute paths via rclone arguments. 6658 Since mounting requires the \f[C]fusermount\f[R] program, rclone will 6659 use the fallback PATH of \f[C]/bin:/usr/bin\f[R] in this scenario. 6660 Please ensure that \f[C]fusermount\f[R] is present on this PATH. 6661 .SS Rclone as Unix mount helper 6662 .PP 6663 The core Unix program \f[C]/bin/mount\f[R] normally takes the 6664 \f[C]-t FSTYPE\f[R] argument then runs the \f[C]/sbin/mount.FSTYPE\f[R] 6665 helper program passing it mount options as \f[C]-o key=val,...\f[R] or 6666 \f[C]--opt=...\f[R]. 6667 Automount (classic or systemd) behaves in a similar way. 6668 .PP 6669 rclone by default expects GNU-style flags \f[C]--key val\f[R]. 6670 To run it as a mount helper you should symlink rclone binary to 6671 \f[C]/sbin/mount.rclone\f[R] and optionally \f[C]/usr/bin/rclonefs\f[R], 6672 e.g. 6673 \f[C]ln -s /usr/bin/rclone /sbin/mount.rclone\f[R]. 6674 rclone will detect it and translate command-line arguments 6675 appropriately. 6676 .PP 6677 Now you can run classic mounts like this: 6678 .IP 6679 .nf 6680 \f[C] 6681 mount sftp1:subdir /mnt/data -t rclone -o vfs_cache_mode=writes,sftp_key_file=/path/to/pem 6682 \f[R] 6683 .fi 6684 .PP 6685 or create systemd mount units: 6686 .IP 6687 .nf 6688 \f[C] 6689 # /etc/systemd/system/mnt-data.mount 6690 [Unit] 6691 Description=Mount for /mnt/data 6692 [Mount] 6693 Type=rclone 6694 What=sftp1:subdir 6695 Where=/mnt/data 6696 Options=rw,_netdev,allow_other,args2env,vfs-cache-mode=writes,config=/etc/rclone.conf,cache-dir=/var/rclone 6697 \f[R] 6698 .fi 6699 .PP 6700 optionally accompanied by systemd automount unit 6701 .IP 6702 .nf 6703 \f[C] 6704 # /etc/systemd/system/mnt-data.automount 6705 [Unit] 6706 Description=AutoMount for /mnt/data 6707 [Automount] 6708 Where=/mnt/data 6709 TimeoutIdleSec=600 6710 [Install] 6711 WantedBy=multi-user.target 6712 \f[R] 6713 .fi 6714 .PP 6715 or add in \f[C]/etc/fstab\f[R] a line like 6716 .IP 6717 .nf 6718 \f[C] 6719 sftp1:subdir /mnt/data rclone rw,noauto,nofail,_netdev,x-systemd.automount,args2env,vfs_cache_mode=writes,config=/etc/rclone.conf,cache_dir=/var/cache/rclone 0 0 6720 \f[R] 6721 .fi 6722 .PP 6723 or use classic Automountd. 6724 Remember to provide explicit \f[C]config=...,cache-dir=...\f[R] as a 6725 workaround for mount units being run without \f[C]HOME\f[R]. 6726 .PP 6727 Rclone in the mount helper mode will split \f[C]-o\f[R] argument(s) by 6728 comma, replace \f[C]_\f[R] by \f[C]-\f[R] and prepend \f[C]--\f[R] to 6729 get the command-line flags. 6730 Options containing commas or spaces can be wrapped in single or double 6731 quotes. 6732 Any inner quotes inside outer quotes of the same type should be doubled. 6733 .PP 6734 Mount option syntax includes a few extra options treated specially: 6735 .IP \[bu] 2 6736 \f[C]env.NAME=VALUE\f[R] will set an environment variable for the mount 6737 process. 6738 This helps with Automountd and Systemd.mount which don\[aq]t allow 6739 setting custom environment for mount helpers. 6740 Typically you will use \f[C]env.HTTPS_PROXY=proxy.host:3128\f[R] or 6741 \f[C]env.HOME=/root\f[R] 6742 .IP \[bu] 2 6743 \f[C]command=cmount\f[R] can be used to run \f[C]cmount\f[R] or any 6744 other rclone command rather than the default \f[C]mount\f[R]. 6745 .IP \[bu] 2 6746 \f[C]args2env\f[R] will pass mount options to the mount helper running 6747 in background via environment variables instead of command line 6748 arguments. 6749 This allows to hide secrets from such commands as \f[C]ps\f[R] or 6750 \f[C]pgrep\f[R]. 6751 .IP \[bu] 2 6752 \f[C]vv...\f[R] will be transformed into appropriate 6753 \f[C]--verbose=N\f[R] 6754 .IP \[bu] 2 6755 standard mount options like \f[C]x-systemd.automount\f[R], 6756 \f[C]_netdev\f[R], \f[C]nosuid\f[R] and alike are intended only for 6757 Automountd and ignored by rclone. 6758 ## VFS - Virtual File System 6759 .PP 6760 This command uses the VFS layer. 6761 This adapts the cloud storage objects that rclone uses into something 6762 which looks much more like a disk filing system. 6763 .PP 6764 Cloud storage objects have lots of properties which aren\[aq]t like disk 6765 files - you can\[aq]t extend them or write to the middle of them, so the 6766 VFS layer has to deal with that. 6767 Because there is no one right way of doing this there are various 6768 options explained below. 6769 .PP 6770 The VFS layer also implements a directory cache - this caches info about 6771 files and directories (but not the data) in memory. 6772 .SS VFS Directory Cache 6773 .PP 6774 Using the \f[C]--dir-cache-time\f[R] flag, you can control how long a 6775 directory should be considered up to date and not refreshed from the 6776 backend. 6777 Changes made through the VFS will appear immediately or invalidate the 6778 cache. 6779 .IP 6780 .nf 6781 \f[C] 6782 --dir-cache-time duration Time to cache directory entries for (default 5m0s) 6783 --poll-interval duration Time to wait between polling for changes. Must be smaller than dir-cache-time. Only on supported remotes. Set to 0 to disable (default 1m0s) 6784 \f[R] 6785 .fi 6786 .PP 6787 However, changes made directly on the cloud storage by the web interface 6788 or a different copy of rclone will only be picked up once the directory 6789 cache expires if the backend configured does not support polling for 6790 changes. 6791 If the backend supports polling, changes will be picked up within the 6792 polling interval. 6793 .PP 6794 You can send a \f[C]SIGHUP\f[R] signal to rclone for it to flush all 6795 directory caches, regardless of how old they are. 6796 Assuming only one rclone instance is running, you can reset the cache 6797 like this: 6798 .IP 6799 .nf 6800 \f[C] 6801 kill -SIGHUP $(pidof rclone) 6802 \f[R] 6803 .fi 6804 .PP 6805 If you configure rclone with a remote control then you can use rclone rc 6806 to flush the whole directory cache: 6807 .IP 6808 .nf 6809 \f[C] 6810 rclone rc vfs/forget 6811 \f[R] 6812 .fi 6813 .PP 6814 Or individual files or directories: 6815 .IP 6816 .nf 6817 \f[C] 6818 rclone rc vfs/forget file=path/to/file dir=path/to/dir 6819 \f[R] 6820 .fi 6821 .SS VFS File Buffering 6822 .PP 6823 The \f[C]--buffer-size\f[R] flag determines the amount of memory, that 6824 will be used to buffer data in advance. 6825 .PP 6826 Each open file will try to keep the specified amount of data in memory 6827 at all times. 6828 The buffered data is bound to one open file and won\[aq]t be shared. 6829 .PP 6830 This flag is a upper limit for the used memory per open file. 6831 The buffer will only use memory for data that is downloaded but not not 6832 yet read. 6833 If the buffer is empty, only a small amount of memory will be used. 6834 .PP 6835 The maximum memory used by rclone for buffering can be up to 6836 \f[C]--buffer-size * open files\f[R]. 6837 .SS VFS File Caching 6838 .PP 6839 These flags control the VFS file caching options. 6840 File caching is necessary to make the VFS layer appear compatible with a 6841 normal file system. 6842 It can be disabled at the cost of some compatibility. 6843 .PP 6844 For example you\[aq]ll need to enable VFS caching if you want to read 6845 and write simultaneously to a file. 6846 See below for more details. 6847 .PP 6848 Note that the VFS cache is separate from the cache backend and you may 6849 find that you need one or the other or both. 6850 .IP 6851 .nf 6852 \f[C] 6853 --cache-dir string Directory rclone will use for caching. 6854 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off) 6855 --vfs-cache-max-age duration Max time since last access of objects in the cache (default 1h0m0s) 6856 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache (default off) 6857 --vfs-cache-min-free-space SizeSuffix Target minimum free space on the disk containing the cache (default off) 6858 --vfs-cache-poll-interval duration Interval to poll the cache for stale objects (default 1m0s) 6859 --vfs-write-back duration Time to writeback files after last use when using cache (default 5s) 6860 \f[R] 6861 .fi 6862 .PP 6863 If run with \f[C]-vv\f[R] rclone will print the location of the file 6864 cache. 6865 The files are stored in the user cache file area which is OS dependent 6866 but can be controlled with \f[C]--cache-dir\f[R] or setting the 6867 appropriate environment variable. 6868 .PP 6869 The cache has 4 different modes selected by \f[C]--vfs-cache-mode\f[R]. 6870 The higher the cache mode the more compatible rclone becomes at the cost 6871 of using disk space. 6872 .PP 6873 Note that files are written back to the remote only when they are closed 6874 and if they haven\[aq]t been accessed for \f[C]--vfs-write-back\f[R] 6875 seconds. 6876 If rclone is quit or dies with files that haven\[aq]t been uploaded, 6877 these will be uploaded next time rclone is run with the same flags. 6878 .PP 6879 If using \f[C]--vfs-cache-max-size\f[R] or 6880 \f[C]--vfs-cache-min-free-size\f[R] note that the cache may exceed these 6881 quotas for two reasons. 6882 Firstly because it is only checked every 6883 \f[C]--vfs-cache-poll-interval\f[R]. 6884 Secondly because open files cannot be evicted from the cache. 6885 When \f[C]--vfs-cache-max-size\f[R] or 6886 \f[C]--vfs-cache-min-free-size\f[R] is exceeded, rclone will attempt to 6887 evict the least accessed files from the cache first. 6888 rclone will start with files that haven\[aq]t been accessed for the 6889 longest. 6890 This cache flushing strategy is efficient and more relevant files are 6891 likely to remain cached. 6892 .PP 6893 The \f[C]--vfs-cache-max-age\f[R] will evict files from the cache after 6894 the set time since last access has passed. 6895 The default value of 1 hour will start evicting files from cache that 6896 haven\[aq]t been accessed for 1 hour. 6897 When a cached file is accessed the 1 hour timer is reset to 0 and will 6898 wait for 1 more hour before evicting. 6899 Specify the time with standard notation, s, m, h, d, w . 6900 .PP 6901 You \f[B]should not\f[R] run two copies of rclone using the same VFS 6902 cache with the same or overlapping remotes if using 6903 \f[C]--vfs-cache-mode > off\f[R]. 6904 This can potentially cause data corruption if you do. 6905 You can work around this by giving each rclone its own cache hierarchy 6906 with \f[C]--cache-dir\f[R]. 6907 You don\[aq]t need to worry about this if the remotes in use don\[aq]t 6908 overlap. 6909 .SS --vfs-cache-mode off 6910 .PP 6911 In this mode (the default) the cache will read directly from the remote 6912 and write directly to the remote without caching anything on disk. 6913 .PP 6914 This will mean some operations are not possible 6915 .IP \[bu] 2 6916 Files can\[aq]t be opened for both read AND write 6917 .IP \[bu] 2 6918 Files opened for write can\[aq]t be seeked 6919 .IP \[bu] 2 6920 Existing files opened for write must have O_TRUNC set 6921 .IP \[bu] 2 6922 Files open for read with O_TRUNC will be opened write only 6923 .IP \[bu] 2 6924 Files open for write only will behave as if O_TRUNC was supplied 6925 .IP \[bu] 2 6926 Open modes O_APPEND, O_TRUNC are ignored 6927 .IP \[bu] 2 6928 If an upload fails it can\[aq]t be retried 6929 .SS --vfs-cache-mode minimal 6930 .PP 6931 This is very similar to \[dq]off\[dq] except that files opened for read 6932 AND write will be buffered to disk. 6933 This means that files opened for write will be a lot more compatible, 6934 but uses the minimal disk space. 6935 .PP 6936 These operations are not possible 6937 .IP \[bu] 2 6938 Files opened for write only can\[aq]t be seeked 6939 .IP \[bu] 2 6940 Existing files opened for write must have O_TRUNC set 6941 .IP \[bu] 2 6942 Files opened for write only will ignore O_APPEND, O_TRUNC 6943 .IP \[bu] 2 6944 If an upload fails it can\[aq]t be retried 6945 .SS --vfs-cache-mode writes 6946 .PP 6947 In this mode files opened for read only are still read directly from the 6948 remote, write only and read/write files are buffered to disk first. 6949 .PP 6950 This mode should support all normal file system operations. 6951 .PP 6952 If an upload fails it will be retried at exponentially increasing 6953 intervals up to 1 minute. 6954 .SS --vfs-cache-mode full 6955 .PP 6956 In this mode all reads and writes are buffered to and from disk. 6957 When data is read from the remote this is buffered to disk as well. 6958 .PP 6959 In this mode the files in the cache will be sparse files and rclone will 6960 keep track of which bits of the files it has downloaded. 6961 .PP 6962 So if an application only reads the starts of each file, then rclone 6963 will only buffer the start of the file. 6964 These files will appear to be their full size in the cache, but they 6965 will be sparse files with only the data that has been downloaded present 6966 in them. 6967 .PP 6968 This mode should support all normal file system operations and is 6969 otherwise identical to \f[C]--vfs-cache-mode\f[R] writes. 6970 .PP 6971 When reading a file rclone will read \f[C]--buffer-size\f[R] plus 6972 \f[C]--vfs-read-ahead\f[R] bytes ahead. 6973 The \f[C]--buffer-size\f[R] is buffered in memory whereas the 6974 \f[C]--vfs-read-ahead\f[R] is buffered on disk. 6975 .PP 6976 When using this mode it is recommended that \f[C]--buffer-size\f[R] is 6977 not set too large and \f[C]--vfs-read-ahead\f[R] is set large if 6978 required. 6979 .PP 6980 \f[B]IMPORTANT\f[R] not all file systems support sparse files. 6981 In particular FAT/exFAT do not. 6982 Rclone will perform very badly if the cache directory is on a filesystem 6983 which doesn\[aq]t support sparse files and it will log an ERROR message 6984 if one is detected. 6985 .SS Fingerprinting 6986 .PP 6987 Various parts of the VFS use fingerprinting to see if a local file copy 6988 has changed relative to a remote file. 6989 Fingerprints are made from: 6990 .IP \[bu] 2 6991 size 6992 .IP \[bu] 2 6993 modification time 6994 .IP \[bu] 2 6995 hash 6996 .PP 6997 where available on an object. 6998 .PP 6999 On some backends some of these attributes are slow to read (they take an 7000 extra API call per object, or extra work per object). 7001 .PP 7002 For example \f[C]hash\f[R] is slow with the \f[C]local\f[R] and 7003 \f[C]sftp\f[R] backends as they have to read the entire file and hash 7004 it, and \f[C]modtime\f[R] is slow with the \f[C]s3\f[R], 7005 \f[C]swift\f[R], \f[C]ftp\f[R] and \f[C]qinqstor\f[R] backends because 7006 they need to do an extra API call to fetch it. 7007 .PP 7008 If you use the \f[C]--vfs-fast-fingerprint\f[R] flag then rclone will 7009 not include the slow operations in the fingerprint. 7010 This makes the fingerprinting less accurate but much faster and will 7011 improve the opening time of cached files. 7012 .PP 7013 If you are running a vfs cache over \f[C]local\f[R], \f[C]s3\f[R] or 7014 \f[C]swift\f[R] backends then using this flag is recommended. 7015 .PP 7016 Note that if you change the value of this flag, the fingerprints of the 7017 files in the cache may be invalidated and the files will need to be 7018 downloaded again. 7019 .SS VFS Chunked Reading 7020 .PP 7021 When rclone reads files from a remote it reads them in chunks. 7022 This means that rather than requesting the whole file rclone reads the 7023 chunk specified. 7024 This can reduce the used download quota for some remotes by requesting 7025 only chunks from the remote that are actually read, at the cost of an 7026 increased number of requests. 7027 .PP 7028 These flags control the chunking: 7029 .IP 7030 .nf 7031 \f[C] 7032 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks (default 128M) 7033 --vfs-read-chunk-size-limit SizeSuffix Max chunk doubling size (default off) 7034 \f[R] 7035 .fi 7036 .PP 7037 Rclone will start reading a chunk of size 7038 \f[C]--vfs-read-chunk-size\f[R], and then double the size for each read. 7039 When \f[C]--vfs-read-chunk-size-limit\f[R] is specified, and greater 7040 than \f[C]--vfs-read-chunk-size\f[R], the chunk size for each open file 7041 will get doubled only until the specified value is reached. 7042 If the value is \[dq]off\[dq], which is the default, the limit is 7043 disabled and the chunk size will grow indefinitely. 7044 .PP 7045 With \f[C]--vfs-read-chunk-size 100M\f[R] and 7046 \f[C]--vfs-read-chunk-size-limit 0\f[R] the following parts will be 7047 downloaded: 0-100M, 100M-200M, 200M-300M, 300M-400M and so on. 7048 When \f[C]--vfs-read-chunk-size-limit 500M\f[R] is specified, the result 7049 would be 0-100M, 100M-300M, 300M-700M, 700M-1200M, 1200M-1700M and so 7050 on. 7051 .PP 7052 Setting \f[C]--vfs-read-chunk-size\f[R] to \f[C]0\f[R] or \[dq]off\[dq] 7053 disables chunked reading. 7054 .SS VFS Performance 7055 .PP 7056 These flags may be used to enable/disable features of the VFS for 7057 performance or other reasons. 7058 See also the chunked reading feature. 7059 .PP 7060 In particular S3 and Swift benefit hugely from the 7061 \f[C]--no-modtime\f[R] flag (or use \f[C]--use-server-modtime\f[R] for a 7062 slightly different effect) as each read of the modification time takes a 7063 transaction. 7064 .IP 7065 .nf 7066 \f[C] 7067 --no-checksum Don\[aq]t compare checksums on up/download. 7068 --no-modtime Don\[aq]t read/write the modification time (can speed things up). 7069 --no-seek Don\[aq]t allow seeking in files. 7070 --read-only Only allow read-only access. 7071 \f[R] 7072 .fi 7073 .PP 7074 Sometimes rclone is delivered reads or writes out of order. 7075 Rather than seeking rclone will wait a short time for the in sequence 7076 read or write to come in. 7077 These flags only come into effect when not using an on disk cache file. 7078 .IP 7079 .nf 7080 \f[C] 7081 --vfs-read-wait duration Time to wait for in-sequence read before seeking (default 20ms) 7082 --vfs-write-wait duration Time to wait for in-sequence write before giving error (default 1s) 7083 \f[R] 7084 .fi 7085 .PP 7086 When using VFS write caching (\f[C]--vfs-cache-mode\f[R] with value 7087 writes or full), the global flag \f[C]--transfers\f[R] can be set to 7088 adjust the number of parallel uploads of modified files from the cache 7089 (the related global flag \f[C]--checkers\f[R] has no effect on the VFS). 7090 .IP 7091 .nf 7092 \f[C] 7093 --transfers int Number of file transfers to run in parallel (default 4) 7094 \f[R] 7095 .fi 7096 .SS VFS Case Sensitivity 7097 .PP 7098 Linux file systems are case-sensitive: two files can differ only by 7099 case, and the exact case must be used when opening a file. 7100 .PP 7101 File systems in modern Windows are case-insensitive but case-preserving: 7102 although existing files can be opened using any case, the exact case 7103 used to create the file is preserved and available for programs to 7104 query. 7105 It is not allowed for two files in the same directory to differ only by 7106 case. 7107 .PP 7108 Usually file systems on macOS are case-insensitive. 7109 It is possible to make macOS file systems case-sensitive but that is not 7110 the default. 7111 .PP 7112 The \f[C]--vfs-case-insensitive\f[R] VFS flag controls how rclone 7113 handles these two cases. 7114 If its value is \[dq]false\[dq], rclone passes file names to the remote 7115 as-is. 7116 If the flag is \[dq]true\[dq] (or appears without a value on the command 7117 line), rclone may perform a \[dq]fixup\[dq] as explained below. 7118 .PP 7119 The user may specify a file name to open/delete/rename/etc with a case 7120 different than what is stored on the remote. 7121 If an argument refers to an existing file with exactly the same name, 7122 then the case of the existing file on the disk will be used. 7123 However, if a file name with exactly the same name is not found but a 7124 name differing only by case exists, rclone will transparently fixup the 7125 name. 7126 This fixup happens only when an existing file is requested. 7127 Case sensitivity of file names created anew by rclone is controlled by 7128 the underlying remote. 7129 .PP 7130 Note that case sensitivity of the operating system running rclone (the 7131 target) may differ from case sensitivity of a file system presented by 7132 rclone (the source). 7133 The flag controls whether \[dq]fixup\[dq] is performed to satisfy the 7134 target. 7135 .PP 7136 If the flag is not provided on the command line, then its default value 7137 depends on the operating system where rclone runs: \[dq]true\[dq] on 7138 Windows and macOS, \[dq]false\[dq] otherwise. 7139 If the flag is provided without a value, then it is \[dq]true\[dq]. 7140 .PP 7141 The \f[C]--no-unicode-normalization\f[R] flag controls whether a similar 7142 \[dq]fixup\[dq] is performed for filenames that differ but are 7143 canonically 7144 equivalent (https://en.wikipedia.org/wiki/Unicode_equivalence) with 7145 respect to unicode. 7146 Unicode normalization can be particularly helpful for users of macOS, 7147 which prefers form NFD instead of the NFC used by most other platforms. 7148 It is therefore highly recommended to keep the default of 7149 \f[C]false\f[R] on macOS, to avoid encoding compatibility issues. 7150 .PP 7151 In the (probably unlikely) event that a directory has multiple duplicate 7152 filenames after applying case and unicode normalization, the 7153 \f[C]--vfs-block-norm-dupes\f[R] flag allows hiding these duplicates. 7154 This comes with a performance tradeoff, as rclone will have to scan the 7155 entire directory for duplicates when listing a directory. 7156 For this reason, it is recommended to leave this disabled if not needed. 7157 However, macOS users may wish to consider using it, as otherwise, if a 7158 remote directory contains both NFC and NFD versions of the same 7159 filename, an odd situation will occur: both versions of the file will be 7160 visible in the mount, and both will appear to be editable, however, 7161 editing either version will actually result in only the NFD version 7162 getting edited under the hood. 7163 \f[C]--vfs-block- norm-dupes\f[R] prevents this confusion by detecting 7164 this scenario, hiding the duplicates, and logging an error, similar to 7165 how this is handled in \f[C]rclone sync\f[R]. 7166 .SS VFS Disk Options 7167 .PP 7168 This flag allows you to manually set the statistics about the filing 7169 system. 7170 It can be useful when those statistics cannot be read correctly 7171 automatically. 7172 .IP 7173 .nf 7174 \f[C] 7175 --vfs-disk-space-total-size Manually set the total disk space size (example: 256G, default: -1) 7176 \f[R] 7177 .fi 7178 .SS Alternate report of used bytes 7179 .PP 7180 Some backends, most notably S3, do not report the amount of bytes used. 7181 If you need this information to be available when running \f[C]df\f[R] 7182 on the filesystem, then pass the flag \f[C]--vfs-used-is-size\f[R] to 7183 rclone. 7184 With this flag set, instead of relying on the backend to report this 7185 information, rclone will scan the whole remote similar to 7186 \f[C]rclone size\f[R] and compute the total used space itself. 7187 .PP 7188 \f[I]WARNING.\f[R] Contrary to \f[C]rclone size\f[R], this flag ignores 7189 filters so that the result is accurate. 7190 However, this is very inefficient and may cost lots of API calls 7191 resulting in extra charges. 7192 Use it as a last resort and only with caching. 7193 .IP 7194 .nf 7195 \f[C] 7196 rclone mount remote:path /path/to/mountpoint [flags] 7197 \f[R] 7198 .fi 7199 .SS Options 7200 .IP 7201 .nf 7202 \f[C] 7203 --allow-non-empty Allow mounting over a non-empty directory (not supported on Windows) 7204 --allow-other Allow access to other users (not supported on Windows) 7205 --allow-root Allow access to root user (not supported on Windows) 7206 --async-read Use asynchronous reads (not supported on Windows) (default true) 7207 --attr-timeout Duration Time for which file/directory attributes are cached (default 1s) 7208 --daemon Run mount in background and exit parent process (as background output is suppressed, use --log-file with --log-format=pid,... to monitor) (not supported on Windows) 7209 --daemon-timeout Duration Time limit for rclone to respond to kernel (not supported on Windows) (default 0s) 7210 --daemon-wait Duration Time to wait for ready mount from daemon (maximum time on Linux, constant sleep time on OSX/BSD) (not supported on Windows) (default 1m0s) 7211 --debug-fuse Debug the FUSE internals - needs -v 7212 --default-permissions Makes kernel enforce access control based on the file mode (not supported on Windows) 7213 --devname string Set the device name - default is remote:path 7214 --dir-cache-time Duration Time to cache directory entries for (default 5m0s) 7215 --dir-perms FileMode Directory permissions (default 0777) 7216 --file-perms FileMode File permissions (default 0666) 7217 --fuse-flag stringArray Flags or arguments to be passed direct to libfuse/WinFsp (repeat if required) 7218 --gid uint32 Override the gid field set by the filesystem (not supported on Windows) (default 1000) 7219 -h, --help help for mount 7220 --max-read-ahead SizeSuffix The number of bytes that can be prefetched for sequential reads (not supported on Windows) (default 128Ki) 7221 --mount-case-insensitive Tristate Tell the OS the mount is case insensitive (true) or sensitive (false) regardless of the backend (auto) (default unset) 7222 --network-mode Mount as remote network drive, instead of fixed disk drive (supported on Windows only) 7223 --no-checksum Don\[aq]t compare checksums on up/download 7224 --no-modtime Don\[aq]t read/write the modification time (can speed things up) 7225 --no-seek Don\[aq]t allow seeking in files 7226 --noappledouble Ignore Apple Double (._) and .DS_Store files (supported on OSX only) (default true) 7227 --noapplexattr Ignore all \[dq]com.apple.*\[dq] extended attributes (supported on OSX only) 7228 -o, --option stringArray Option for libfuse/WinFsp (repeat if required) 7229 --poll-interval Duration Time to wait between polling for changes, must be smaller than dir-cache-time and only on supported remotes (set 0 to disable) (default 1m0s) 7230 --read-only Only allow read-only access 7231 --uid uint32 Override the uid field set by the filesystem (not supported on Windows) (default 1000) 7232 --umask int Override the permission bits set by the filesystem (not supported on Windows) (default 2) 7233 --vfs-block-norm-dupes If duplicate filenames exist in the same directory (after normalization), log an error and hide the duplicates (may have a performance cost) 7234 --vfs-cache-max-age Duration Max time since last access of objects in the cache (default 1h0m0s) 7235 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache (default off) 7236 --vfs-cache-min-free-space SizeSuffix Target minimum free space on the disk containing the cache (default off) 7237 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off) 7238 --vfs-cache-poll-interval Duration Interval to poll the cache for stale objects (default 1m0s) 7239 --vfs-case-insensitive If a file name not found, find a case insensitive match 7240 --vfs-disk-space-total-size SizeSuffix Specify the total space of disk (default off) 7241 --vfs-fast-fingerprint Use fast (less accurate) fingerprints for change detection 7242 --vfs-read-ahead SizeSuffix Extra read ahead over --buffer-size when using cache-mode full 7243 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks (default 128Mi) 7244 --vfs-read-chunk-size-limit SizeSuffix If greater than --vfs-read-chunk-size, double the chunk size after each chunk read, until the limit is reached (\[aq]off\[aq] is unlimited) (default off) 7245 --vfs-read-wait Duration Time to wait for in-sequence read before seeking (default 20ms) 7246 --vfs-refresh Refreshes the directory cache recursively in the background on start 7247 --vfs-used-is-size rclone size Use the rclone size algorithm for Used size 7248 --vfs-write-back Duration Time to writeback files after last use when using cache (default 5s) 7249 --vfs-write-wait Duration Time to wait for in-sequence write before giving error (default 1s) 7250 --volname string Set the volume name (supported on Windows and OSX only) 7251 --write-back-cache Makes kernel buffer writes before sending them to rclone (without this, writethrough caching is used) (not supported on Windows) 7252 \f[R] 7253 .fi 7254 .SS Filter Options 7255 .PP 7256 Flags for filtering directory listings. 7257 .IP 7258 .nf 7259 \f[C] 7260 --delete-excluded Delete files on dest excluded from sync 7261 --exclude stringArray Exclude files matching pattern 7262 --exclude-from stringArray Read file exclude patterns from file (use - to read from stdin) 7263 --exclude-if-present stringArray Exclude directories if filename is present 7264 --files-from stringArray Read list of source-file names from file (use - to read from stdin) 7265 --files-from-raw stringArray Read list of source-file names from file without any processing of lines (use - to read from stdin) 7266 -f, --filter stringArray Add a file filtering rule 7267 --filter-from stringArray Read file filtering patterns from a file (use - to read from stdin) 7268 --ignore-case Ignore case in filters (case insensitive) 7269 --include stringArray Include files matching pattern 7270 --include-from stringArray Read file include patterns from file (use - to read from stdin) 7271 --max-age Duration Only transfer files younger than this in s or suffix ms|s|m|h|d|w|M|y (default off) 7272 --max-depth int If set limits the recursion depth to this (default -1) 7273 --max-size SizeSuffix Only transfer files smaller than this in KiB or suffix B|K|M|G|T|P (default off) 7274 --metadata-exclude stringArray Exclude metadatas matching pattern 7275 --metadata-exclude-from stringArray Read metadata exclude patterns from file (use - to read from stdin) 7276 --metadata-filter stringArray Add a metadata filtering rule 7277 --metadata-filter-from stringArray Read metadata filtering patterns from a file (use - to read from stdin) 7278 --metadata-include stringArray Include metadatas matching pattern 7279 --metadata-include-from stringArray Read metadata include patterns from file (use - to read from stdin) 7280 --min-age Duration Only transfer files older than this in s or suffix ms|s|m|h|d|w|M|y (default off) 7281 --min-size SizeSuffix Only transfer files bigger than this in KiB or suffix B|K|M|G|T|P (default off) 7282 \f[R] 7283 .fi 7284 .PP 7285 See the global flags page (https://rclone.org/flags/) for global options 7286 not listed here. 7287 .SH SEE ALSO 7288 .IP \[bu] 2 7289 rclone (https://rclone.org/commands/rclone/) - Show help for rclone 7290 commands, flags and backends. 7291 .SH rclone moveto 7292 .PP 7293 Move file or directory from source to dest. 7294 .SS Synopsis 7295 .PP 7296 If source:path is a file or directory then it moves it to a file or 7297 directory named dest:path. 7298 .PP 7299 This can be used to rename files or upload single files to other than 7300 their existing name. 7301 If the source is a directory then it acts exactly like the 7302 move (https://rclone.org/commands/rclone_move/) command. 7303 .PP 7304 So 7305 .IP 7306 .nf 7307 \f[C] 7308 rclone moveto src dst 7309 \f[R] 7310 .fi 7311 .PP 7312 where src and dst are rclone paths, either remote:path or /path/to/local 7313 or C:. 7314 .PP 7315 This will: 7316 .IP 7317 .nf 7318 \f[C] 7319 if src is file 7320 move it to dst, overwriting an existing file if it exists 7321 if src is directory 7322 move it to dst, overwriting existing files if they exist 7323 see move command for full details 7324 \f[R] 7325 .fi 7326 .PP 7327 This doesn\[aq]t transfer files that are identical on src and dst, 7328 testing by size and modification time or MD5SUM. 7329 src will be deleted on successful transfer. 7330 .PP 7331 \f[B]Important\f[R]: Since this can cause data loss, test first with the 7332 \f[C]--dry-run\f[R] or the \f[C]--interactive\f[R]/\f[C]-i\f[R] flag. 7333 .PP 7334 \f[B]Note\f[R]: Use the \f[C]-P\f[R]/\f[C]--progress\f[R] flag to view 7335 real-time transfer statistics. 7336 .IP 7337 .nf 7338 \f[C] 7339 rclone moveto source:path dest:path [flags] 7340 \f[R] 7341 .fi 7342 .SS Options 7343 .IP 7344 .nf 7345 \f[C] 7346 -h, --help help for moveto 7347 \f[R] 7348 .fi 7349 .SS Copy Options 7350 .PP 7351 Flags for anything which can Copy a file. 7352 .IP 7353 .nf 7354 \f[C] 7355 --check-first Do all the checks before starting transfers 7356 -c, --checksum Check for changes with size & checksum (if available, or fallback to size only). 7357 --compare-dest stringArray Include additional comma separated server-side paths during comparison 7358 --copy-dest stringArray Implies --compare-dest but also copies files from paths into destination 7359 --cutoff-mode HARD|SOFT|CAUTIOUS Mode to stop transfers when reaching the max transfer limit HARD|SOFT|CAUTIOUS (default HARD) 7360 --ignore-case-sync Ignore case when synchronizing 7361 --ignore-checksum Skip post copy check of checksums 7362 --ignore-existing Skip all files that exist on destination 7363 --ignore-size Ignore size when skipping use modtime or checksum 7364 -I, --ignore-times Don\[aq]t skip items that match size and time - transfer all unconditionally 7365 --immutable Do not modify files, fail if existing files have been modified 7366 --inplace Download directly to destination file instead of atomic download to temp/rename 7367 --max-backlog int Maximum number of objects in sync or check backlog (default 10000) 7368 --max-duration Duration Maximum duration rclone will transfer data for (default 0s) 7369 --max-transfer SizeSuffix Maximum size of data to transfer (default off) 7370 -M, --metadata If set, preserve metadata when copying objects 7371 --modify-window Duration Max time diff to be considered the same (default 1ns) 7372 --multi-thread-chunk-size SizeSuffix Chunk size for multi-thread downloads / uploads, if not set by filesystem (default 64Mi) 7373 --multi-thread-cutoff SizeSuffix Use multi-thread downloads for files above this size (default 256Mi) 7374 --multi-thread-streams int Number of streams to use for multi-thread downloads (default 4) 7375 --multi-thread-write-buffer-size SizeSuffix In memory buffer size for writing when in multi-thread mode (default 128Ki) 7376 --no-check-dest Don\[aq]t check the destination, copy regardless 7377 --no-traverse Don\[aq]t traverse destination file system on copy 7378 --no-update-dir-modtime Don\[aq]t update directory modification times 7379 --no-update-modtime Don\[aq]t update destination modtime if files identical 7380 --order-by string Instructions on how to order the transfers, e.g. \[aq]size,descending\[aq] 7381 --partial-suffix string Add partial-suffix to temporary file name when --inplace is not used (default \[dq].partial\[dq]) 7382 --refresh-times Refresh the modtime of remote files 7383 --server-side-across-configs Allow server-side operations (e.g. copy) to work across different configs 7384 --size-only Skip based on size only, not modtime or checksum 7385 --streaming-upload-cutoff SizeSuffix Cutoff for switching to chunked upload if file size is unknown, upload starts after reaching cutoff or when file ends (default 100Ki) 7386 -u, --update Skip files that are newer on the destination 7387 \f[R] 7388 .fi 7389 .SS Important Options 7390 .PP 7391 Important flags useful for most commands. 7392 .IP 7393 .nf 7394 \f[C] 7395 -n, --dry-run Do a trial run with no permanent changes 7396 -i, --interactive Enable interactive mode 7397 -v, --verbose count Print lots more stuff (repeat for more) 7398 \f[R] 7399 .fi 7400 .SS Filter Options 7401 .PP 7402 Flags for filtering directory listings. 7403 .IP 7404 .nf 7405 \f[C] 7406 --delete-excluded Delete files on dest excluded from sync 7407 --exclude stringArray Exclude files matching pattern 7408 --exclude-from stringArray Read file exclude patterns from file (use - to read from stdin) 7409 --exclude-if-present stringArray Exclude directories if filename is present 7410 --files-from stringArray Read list of source-file names from file (use - to read from stdin) 7411 --files-from-raw stringArray Read list of source-file names from file without any processing of lines (use - to read from stdin) 7412 -f, --filter stringArray Add a file filtering rule 7413 --filter-from stringArray Read file filtering patterns from a file (use - to read from stdin) 7414 --ignore-case Ignore case in filters (case insensitive) 7415 --include stringArray Include files matching pattern 7416 --include-from stringArray Read file include patterns from file (use - to read from stdin) 7417 --max-age Duration Only transfer files younger than this in s or suffix ms|s|m|h|d|w|M|y (default off) 7418 --max-depth int If set limits the recursion depth to this (default -1) 7419 --max-size SizeSuffix Only transfer files smaller than this in KiB or suffix B|K|M|G|T|P (default off) 7420 --metadata-exclude stringArray Exclude metadatas matching pattern 7421 --metadata-exclude-from stringArray Read metadata exclude patterns from file (use - to read from stdin) 7422 --metadata-filter stringArray Add a metadata filtering rule 7423 --metadata-filter-from stringArray Read metadata filtering patterns from a file (use - to read from stdin) 7424 --metadata-include stringArray Include metadatas matching pattern 7425 --metadata-include-from stringArray Read metadata include patterns from file (use - to read from stdin) 7426 --min-age Duration Only transfer files older than this in s or suffix ms|s|m|h|d|w|M|y (default off) 7427 --min-size SizeSuffix Only transfer files bigger than this in KiB or suffix B|K|M|G|T|P (default off) 7428 \f[R] 7429 .fi 7430 .SS Listing Options 7431 .PP 7432 Flags for listing directories. 7433 .IP 7434 .nf 7435 \f[C] 7436 --default-time Time Time to show if modtime is unknown for files and directories (default 2000-01-01T00:00:00Z) 7437 --fast-list Use recursive list if available; uses more memory but fewer transactions 7438 \f[R] 7439 .fi 7440 .PP 7441 See the global flags page (https://rclone.org/flags/) for global options 7442 not listed here. 7443 .SH SEE ALSO 7444 .IP \[bu] 2 7445 rclone (https://rclone.org/commands/rclone/) - Show help for rclone 7446 commands, flags and backends. 7447 .SH rclone ncdu 7448 .PP 7449 Explore a remote with a text based user interface. 7450 .SS Synopsis 7451 .PP 7452 This displays a text based user interface allowing the navigation of a 7453 remote. 7454 It is most useful for answering the question - \[dq]What is using all my 7455 disk space?\[dq]. 7456 .PP 7457 To make the user interface it first scans the entire remote given and 7458 builds an in memory representation. 7459 rclone ncdu can be used during this scanning phase and you will see it 7460 building up the directory structure as it goes along. 7461 .PP 7462 You can interact with the user interface using key presses, press 7463 \[aq]?\[aq] to toggle the help on and off. 7464 The supported keys are: 7465 .IP 7466 .nf 7467 \f[C] 7468 \[ua],\[da] or k,j to Move 7469 \[->],l to enter 7470 \[<-],h to return 7471 g toggle graph 7472 c toggle counts 7473 a toggle average size in directory 7474 m toggle modified time 7475 u toggle human-readable format 7476 n,s,C,A,M sort by name,size,count,asize,mtime 7477 d delete file/directory 7478 v select file/directory 7479 V enter visual select mode 7480 D delete selected files/directories 7481 y copy current path to clipboard 7482 Y display current path 7483 \[ha]L refresh screen (fix screen corruption) 7484 r recalculate file sizes 7485 ? to toggle help on and off 7486 q/ESC/\[ha]c to quit 7487 \f[R] 7488 .fi 7489 .PP 7490 Listed files/directories may be prefixed by a one-character flag, some 7491 of them combined with a description in brackets at end of line. 7492 These flags have the following meaning: 7493 .IP 7494 .nf 7495 \f[C] 7496 e means this is an empty directory, i.e. contains no files (but 7497 may contain empty subdirectories) 7498 \[ti] means this is a directory where some of the files (possibly in 7499 subdirectories) have unknown size, and therefore the directory 7500 size may be underestimated (and average size inaccurate, as it 7501 is average of the files with known sizes). 7502 \&. means an error occurred while reading a subdirectory, and 7503 therefore the directory size may be underestimated (and average 7504 size inaccurate) 7505 ! means an error occurred while reading this directory 7506 \f[R] 7507 .fi 7508 .PP 7509 This an homage to the ncdu tool (https://dev.yorhel.nl/ncdu) but for 7510 rclone remotes. 7511 It is missing lots of features at the moment but is useful as it stands. 7512 .PP 7513 Note that it might take some time to delete big files/directories. 7514 The UI won\[aq]t respond in the meantime since the deletion is done 7515 synchronously. 7516 .PP 7517 For a non-interactive listing of the remote, see the 7518 tree (https://rclone.org/commands/rclone_tree/) command. 7519 To just get the total size of the remote you can also use the 7520 size (https://rclone.org/commands/rclone_size/) command. 7521 .IP 7522 .nf 7523 \f[C] 7524 rclone ncdu remote:path [flags] 7525 \f[R] 7526 .fi 7527 .SS Options 7528 .IP 7529 .nf 7530 \f[C] 7531 -h, --help help for ncdu 7532 \f[R] 7533 .fi 7534 .SS Filter Options 7535 .PP 7536 Flags for filtering directory listings. 7537 .IP 7538 .nf 7539 \f[C] 7540 --delete-excluded Delete files on dest excluded from sync 7541 --exclude stringArray Exclude files matching pattern 7542 --exclude-from stringArray Read file exclude patterns from file (use - to read from stdin) 7543 --exclude-if-present stringArray Exclude directories if filename is present 7544 --files-from stringArray Read list of source-file names from file (use - to read from stdin) 7545 --files-from-raw stringArray Read list of source-file names from file without any processing of lines (use - to read from stdin) 7546 -f, --filter stringArray Add a file filtering rule 7547 --filter-from stringArray Read file filtering patterns from a file (use - to read from stdin) 7548 --ignore-case Ignore case in filters (case insensitive) 7549 --include stringArray Include files matching pattern 7550 --include-from stringArray Read file include patterns from file (use - to read from stdin) 7551 --max-age Duration Only transfer files younger than this in s or suffix ms|s|m|h|d|w|M|y (default off) 7552 --max-depth int If set limits the recursion depth to this (default -1) 7553 --max-size SizeSuffix Only transfer files smaller than this in KiB or suffix B|K|M|G|T|P (default off) 7554 --metadata-exclude stringArray Exclude metadatas matching pattern 7555 --metadata-exclude-from stringArray Read metadata exclude patterns from file (use - to read from stdin) 7556 --metadata-filter stringArray Add a metadata filtering rule 7557 --metadata-filter-from stringArray Read metadata filtering patterns from a file (use - to read from stdin) 7558 --metadata-include stringArray Include metadatas matching pattern 7559 --metadata-include-from stringArray Read metadata include patterns from file (use - to read from stdin) 7560 --min-age Duration Only transfer files older than this in s or suffix ms|s|m|h|d|w|M|y (default off) 7561 --min-size SizeSuffix Only transfer files bigger than this in KiB or suffix B|K|M|G|T|P (default off) 7562 \f[R] 7563 .fi 7564 .SS Listing Options 7565 .PP 7566 Flags for listing directories. 7567 .IP 7568 .nf 7569 \f[C] 7570 --default-time Time Time to show if modtime is unknown for files and directories (default 2000-01-01T00:00:00Z) 7571 --fast-list Use recursive list if available; uses more memory but fewer transactions 7572 \f[R] 7573 .fi 7574 .PP 7575 See the global flags page (https://rclone.org/flags/) for global options 7576 not listed here. 7577 .SH SEE ALSO 7578 .IP \[bu] 2 7579 rclone (https://rclone.org/commands/rclone/) - Show help for rclone 7580 commands, flags and backends. 7581 .SH rclone nfsmount 7582 .PP 7583 Mount the remote as file system on a mountpoint. 7584 .SS Synopsis 7585 .PP 7586 rclone nfsmount allows Linux, FreeBSD, macOS and Windows to mount any of 7587 Rclone\[aq]s cloud storage systems as a file system with FUSE. 7588 .PP 7589 First set up your remote using \f[C]rclone config\f[R]. 7590 Check it works with \f[C]rclone ls\f[R] etc. 7591 .PP 7592 On Linux and macOS, you can run mount in either foreground or background 7593 (aka daemon) mode. 7594 Mount runs in foreground mode by default. 7595 Use the \f[C]--daemon\f[R] flag to force background mode. 7596 On Windows you can run mount in foreground only, the flag is ignored. 7597 .PP 7598 In background mode rclone acts as a generic Unix mount program: the main 7599 program starts, spawns background rclone process to setup and maintain 7600 the mount, waits until success or timeout and exits with appropriate 7601 code (killing the child process if it fails). 7602 .PP 7603 On Linux/macOS/FreeBSD start the mount like this, where 7604 \f[C]/path/to/local/mount\f[R] is an \f[B]empty\f[R] \f[B]existing\f[R] 7605 directory: 7606 .IP 7607 .nf 7608 \f[C] 7609 rclone nfsmount remote:path/to/files /path/to/local/mount 7610 \f[R] 7611 .fi 7612 .PP 7613 On Windows you can start a mount in different ways. 7614 See below for details. 7615 If foreground mount is used interactively from a console window, rclone 7616 will serve the mount and occupy the console so another window should be 7617 used to work with the mount until rclone is interrupted e.g. 7618 by pressing Ctrl-C. 7619 .PP 7620 The following examples will mount to an automatically assigned drive, to 7621 specific drive letter \f[C]X:\f[R], to path 7622 \f[C]C:\[rs]path\[rs]parent\[rs]mount\f[R] (where parent directory or 7623 drive must exist, and mount must \f[B]not\f[R] exist, and is not 7624 supported when mounting as a network drive), and the last example will 7625 mount as network share \f[C]\[rs]\[rs]cloud\[rs]remote\f[R] and map it 7626 to an automatically assigned drive: 7627 .IP 7628 .nf 7629 \f[C] 7630 rclone nfsmount remote:path/to/files * 7631 rclone nfsmount remote:path/to/files X: 7632 rclone nfsmount remote:path/to/files C:\[rs]path\[rs]parent\[rs]mount 7633 rclone nfsmount remote:path/to/files \[rs]\[rs]cloud\[rs]remote 7634 \f[R] 7635 .fi 7636 .PP 7637 When the program ends while in foreground mode, either via Ctrl+C or 7638 receiving a SIGINT or SIGTERM signal, the mount should be automatically 7639 stopped. 7640 .PP 7641 When running in background mode the user will have to stop the mount 7642 manually: 7643 .IP 7644 .nf 7645 \f[C] 7646 # Linux 7647 fusermount -u /path/to/local/mount 7648 # OS X 7649 umount /path/to/local/mount 7650 \f[R] 7651 .fi 7652 .PP 7653 The umount operation can fail, for example when the mountpoint is busy. 7654 When that happens, it is the user\[aq]s responsibility to stop the mount 7655 manually. 7656 .PP 7657 The size of the mounted file system will be set according to information 7658 retrieved from the remote, the same as returned by the rclone 7659 about (https://rclone.org/commands/rclone_about/) command. 7660 Remotes with unlimited storage may report the used size only, then an 7661 additional 1 PiB of free space is assumed. 7662 If the remote does not 7663 support (https://rclone.org/overview/#optional-features) the about 7664 feature at all, then 1 PiB is set as both the total and the free size. 7665 .SS Installing on Windows 7666 .PP 7667 To run rclone nfsmount on Windows, you will need to download and install 7668 WinFsp (http://www.secfs.net/winfsp/). 7669 .PP 7670 WinFsp (https://github.com/winfsp/winfsp) is an open-source Windows File 7671 System Proxy which makes it easy to write user space file systems for 7672 Windows. 7673 It provides a FUSE emulation layer which rclone uses combination with 7674 cgofuse (https://github.com/winfsp/cgofuse). 7675 Both of these packages are by Bill Zissimopoulos who was very helpful 7676 during the implementation of rclone nfsmount for Windows. 7677 .SS Mounting modes on windows 7678 .PP 7679 Unlike other operating systems, Microsoft Windows provides a different 7680 filesystem type for network and fixed drives. 7681 It optimises access on the assumption fixed disk drives are fast and 7682 reliable, while network drives have relatively high latency and less 7683 reliability. 7684 Some settings can also be differentiated between the two types, for 7685 example that Windows Explorer should just display icons and not create 7686 preview thumbnails for image and video files on network drives. 7687 .PP 7688 In most cases, rclone will mount the remote as a normal, fixed disk 7689 drive by default. 7690 However, you can also choose to mount it as a remote network drive, 7691 often described as a network share. 7692 If you mount an rclone remote using the default, fixed drive mode and 7693 experience unexpected program errors, freezes or other issues, consider 7694 mounting as a network drive instead. 7695 .PP 7696 When mounting as a fixed disk drive you can either mount to an unused 7697 drive letter, or to a path representing a \f[B]nonexistent\f[R] 7698 subdirectory of an \f[B]existing\f[R] parent directory or drive. 7699 Using the special value \f[C]*\f[R] will tell rclone to automatically 7700 assign the next available drive letter, starting with Z: and moving 7701 backward. 7702 Examples: 7703 .IP 7704 .nf 7705 \f[C] 7706 rclone nfsmount remote:path/to/files * 7707 rclone nfsmount remote:path/to/files X: 7708 rclone nfsmount remote:path/to/files C:\[rs]path\[rs]parent\[rs]mount 7709 rclone nfsmount remote:path/to/files X: 7710 \f[R] 7711 .fi 7712 .PP 7713 Option \f[C]--volname\f[R] can be used to set a custom volume name for 7714 the mounted file system. 7715 The default is to use the remote name and path. 7716 .PP 7717 To mount as network drive, you can add option \f[C]--network-mode\f[R] 7718 to your nfsmount command. 7719 Mounting to a directory path is not supported in this mode, it is a 7720 limitation Windows imposes on junctions, so the remote must always be 7721 mounted to a drive letter. 7722 .IP 7723 .nf 7724 \f[C] 7725 rclone nfsmount remote:path/to/files X: --network-mode 7726 \f[R] 7727 .fi 7728 .PP 7729 A volume name specified with \f[C]--volname\f[R] will be used to create 7730 the network share path. 7731 A complete UNC path, such as \f[C]\[rs]\[rs]cloud\[rs]remote\f[R], 7732 optionally with path 7733 \f[C]\[rs]\[rs]cloud\[rs]remote\[rs]madeup\[rs]path\f[R], will be used 7734 as is. 7735 Any other string will be used as the share part, after a default prefix 7736 \f[C]\[rs]\[rs]server\[rs]\f[R]. 7737 If no volume name is specified then \f[C]\[rs]\[rs]server\[rs]share\f[R] 7738 will be used. 7739 You must make sure the volume name is unique when you are mounting more 7740 than one drive, or else the mount command will fail. 7741 The share name will treated as the volume label for the mapped drive, 7742 shown in Windows Explorer etc, while the complete 7743 \f[C]\[rs]\[rs]server\[rs]share\f[R] will be reported as the remote UNC 7744 path by \f[C]net use\f[R] etc, just like a normal network drive mapping. 7745 .PP 7746 If you specify a full network share UNC path with \f[C]--volname\f[R], 7747 this will implicitly set the \f[C]--network-mode\f[R] option, so the 7748 following two examples have same result: 7749 .IP 7750 .nf 7751 \f[C] 7752 rclone nfsmount remote:path/to/files X: --network-mode 7753 rclone nfsmount remote:path/to/files X: --volname \[rs]\[rs]server\[rs]share 7754 \f[R] 7755 .fi 7756 .PP 7757 You may also specify the network share UNC path as the mountpoint 7758 itself. 7759 Then rclone will automatically assign a drive letter, same as with 7760 \f[C]*\f[R] and use that as mountpoint, and instead use the UNC path 7761 specified as the volume name, as if it were specified with the 7762 \f[C]--volname\f[R] option. 7763 This will also implicitly set the \f[C]--network-mode\f[R] option. 7764 This means the following two examples have same result: 7765 .IP 7766 .nf 7767 \f[C] 7768 rclone nfsmount remote:path/to/files \[rs]\[rs]cloud\[rs]remote 7769 rclone nfsmount remote:path/to/files * --volname \[rs]\[rs]cloud\[rs]remote 7770 \f[R] 7771 .fi 7772 .PP 7773 There is yet another way to enable network mode, and to set the share 7774 path, and that is to pass the \[dq]native\[dq] libfuse/WinFsp option 7775 directly: \f[C]--fuse-flag --VolumePrefix=\[rs]server\[rs]share\f[R]. 7776 Note that the path must be with just a single backslash prefix in this 7777 case. 7778 .PP 7779 \f[I]Note:\f[R] In previous versions of rclone this was the only 7780 supported method. 7781 .PP 7782 Read more about drive 7783 mapping (https://en.wikipedia.org/wiki/Drive_mapping) 7784 .PP 7785 See also Limitations section below. 7786 .SS Windows filesystem permissions 7787 .PP 7788 The FUSE emulation layer on Windows must convert between the POSIX-based 7789 permission model used in FUSE, and the permission model used in Windows, 7790 based on access-control lists (ACL). 7791 .PP 7792 The mounted filesystem will normally get three entries in its 7793 access-control list (ACL), representing permissions for the POSIX 7794 permission scopes: Owner, group and others. 7795 By default, the owner and group will be taken from the current user, and 7796 the built-in group \[dq]Everyone\[dq] will be used to represent others. 7797 The user/group can be customized with FUSE options \[dq]UserName\[dq] 7798 and \[dq]GroupName\[dq], e.g. 7799 \f[C]-o UserName=user123 -o GroupName=\[dq]Authenticated Users\[dq]\f[R]. 7800 The permissions on each entry will be set according to options 7801 \f[C]--dir-perms\f[R] and \f[C]--file-perms\f[R], which takes a value in 7802 traditional Unix numeric 7803 notation (https://en.wikipedia.org/wiki/File-system_permissions#Numeric_notation). 7804 .PP 7805 The default permissions corresponds to 7806 \f[C]--file-perms 0666 --dir-perms 0777\f[R], i.e. 7807 read and write permissions to everyone. 7808 This means you will not be able to start any programs from the mount. 7809 To be able to do that you must add execute permissions, e.g. 7810 \f[C]--file-perms 0777 --dir-perms 0777\f[R] to add it to everyone. 7811 If the program needs to write files, chances are you will have to enable 7812 VFS File Caching as well (see also limitations). 7813 Note that the default write permission have some restrictions for 7814 accounts other than the owner, specifically it lacks the \[dq]write 7815 extended attributes\[dq], as explained next. 7816 .PP 7817 The mapping of permissions is not always trivial, and the result you see 7818 in Windows Explorer may not be exactly like you expected. 7819 For example, when setting a value that includes write access for the 7820 group or others scope, this will be mapped to individual permissions 7821 \[dq]write attributes\[dq], \[dq]write data\[dq] and \[dq]append 7822 data\[dq], but not \[dq]write extended attributes\[dq]. 7823 Windows will then show this as basic permission \[dq]Special\[dq] 7824 instead of \[dq]Write\[dq], because \[dq]Write\[dq] also covers the 7825 \[dq]write extended attributes\[dq] permission. 7826 When setting digit 0 for group or others, to indicate no permissions, 7827 they will still get individual permissions \[dq]read attributes\[dq], 7828 \[dq]read extended attributes\[dq] and \[dq]read permissions\[dq]. 7829 This is done for compatibility reasons, e.g. 7830 to allow users without additional permissions to be able to read basic 7831 metadata about files like in Unix. 7832 .PP 7833 WinFsp 2021 (version 1.9) introduced a new FUSE option 7834 \[dq]FileSecurity\[dq], that allows the complete specification of file 7835 security descriptors using 7836 SDDL (https://docs.microsoft.com/en-us/windows/win32/secauthz/security-descriptor-string-format). 7837 With this you get detailed control of the resulting permissions, 7838 compared to use of the POSIX permissions described above, and no 7839 additional permissions will be added automatically for compatibility 7840 with Unix. 7841 Some example use cases will following. 7842 .PP 7843 If you set POSIX permissions for only allowing access to the owner, 7844 using \f[C]--file-perms 0600 --dir-perms 0700\f[R], the user group and 7845 the built-in \[dq]Everyone\[dq] group will still be given some special 7846 permissions, as described above. 7847 Some programs may then (incorrectly) interpret this as the file being 7848 accessible by everyone, for example an SSH client may warn about 7849 \[dq]unprotected private key file\[dq]. 7850 You can work around this by specifying 7851 \f[C]-o FileSecurity=\[dq]D:P(A;;FA;;;OW)\[dq]\f[R], which sets file all 7852 access (FA) to the owner (OW), and nothing else. 7853 .PP 7854 When setting write permissions then, except for the owner, this does not 7855 include the \[dq]write extended attributes\[dq] permission, as mentioned 7856 above. 7857 This may prevent applications from writing to files, giving permission 7858 denied error instead. 7859 To set working write permissions for the built-in \[dq]Everyone\[dq] 7860 group, similar to what it gets by default but with the addition of the 7861 \[dq]write extended attributes\[dq], you can specify 7862 \f[C]-o FileSecurity=\[dq]D:P(A;;FRFW;;;WD)\[dq]\f[R], which sets file 7863 read (FR) and file write (FW) to everyone (WD). 7864 If file execute (FX) is also needed, then change to 7865 \f[C]-o FileSecurity=\[dq]D:P(A;;FRFWFX;;;WD)\[dq]\f[R], or set file all 7866 access (FA) to get full access permissions, including delete, with 7867 \f[C]-o FileSecurity=\[dq]D:P(A;;FA;;;WD)\[dq]\f[R]. 7868 .SS Windows caveats 7869 .PP 7870 Drives created as Administrator are not visible to other accounts, not 7871 even an account that was elevated to Administrator with the User Account 7872 Control (UAC) feature. 7873 A result of this is that if you mount to a drive letter from a Command 7874 Prompt run as Administrator, and then try to access the same drive from 7875 Windows Explorer (which does not run as Administrator), you will not be 7876 able to see the mounted drive. 7877 .PP 7878 If you don\[aq]t need to access the drive from applications running with 7879 administrative privileges, the easiest way around this is to always 7880 create the mount from a non-elevated command prompt. 7881 .PP 7882 To make mapped drives available to the user account that created them 7883 regardless if elevated or not, there is a special Windows setting called 7884 linked 7885 connections (https://docs.microsoft.com/en-us/troubleshoot/windows-client/networking/mapped-drives-not-available-from-elevated-command#detail-to-configure-the-enablelinkedconnections-registry-entry) 7886 that can be enabled. 7887 .PP 7888 It is also possible to make a drive mount available to everyone on the 7889 system, by running the process creating it as the built-in SYSTEM 7890 account. 7891 There are several ways to do this: One is to use the command-line 7892 utility 7893 PsExec (https://docs.microsoft.com/en-us/sysinternals/downloads/psexec), 7894 from Microsoft\[aq]s Sysinternals suite, which has option \f[C]-s\f[R] 7895 to start processes as the SYSTEM account. 7896 Another alternative is to run the mount command from a Windows Scheduled 7897 Task, or a Windows Service, configured to run as the SYSTEM account. 7898 A third alternative is to use the WinFsp.Launcher 7899 infrastructure (https://github.com/winfsp/winfsp/wiki/WinFsp-Service-Architecture)). 7900 Read more in the install documentation (https://rclone.org/install/). 7901 Note that when running rclone as another user, it will not use the 7902 configuration file from your profile unless you tell it to with the 7903 \f[C]--config\f[R] (https://rclone.org/docs/#config-config-file) option. 7904 Note also that it is now the SYSTEM account that will have the owner 7905 permissions, and other accounts will have permissions according to the 7906 group or others scopes. 7907 As mentioned above, these will then not get the \[dq]write extended 7908 attributes\[dq] permission, and this may prevent writing to files. 7909 You can work around this with the FileSecurity option, see example 7910 above. 7911 .PP 7912 Note that mapping to a directory path, instead of a drive letter, does 7913 not suffer from the same limitations. 7914 .SS Mounting on macOS 7915 .PP 7916 Mounting on macOS can be done either via built-in NFS 7917 server (https://rclone.org/commands/rclone_serve_nfs/), 7918 macFUSE (https://osxfuse.github.io/) (also known as osxfuse) or 7919 FUSE-T (https://www.fuse-t.org/). 7920 macFUSE is a traditional FUSE driver utilizing a macOS kernel extension 7921 (kext). 7922 FUSE-T is an alternative FUSE system which \[dq]mounts\[dq] via an NFSv4 7923 local server. 7924 .SS Unicode Normalization 7925 .PP 7926 It is highly recommended to keep the default of 7927 \f[C]--no-unicode-normalization=false\f[R] for all \f[C]mount\f[R] and 7928 \f[C]serve\f[R] commands on macOS. 7929 For details, see 7930 vfs-case-sensitivity (https://rclone.org/commands/rclone_mount/#vfs-case-sensitivity). 7931 .SS NFS mount 7932 .PP 7933 This method spins up an NFS server using serve 7934 nfs (https://rclone.org/commands/rclone_serve_nfs/) command and mounts 7935 it to the specified mountpoint. 7936 If you run this in background mode using |--daemon|, you will need to 7937 send SIGTERM signal to the rclone process using |kill| command to stop 7938 the mount. 7939 .PP 7940 Note that \f[C]--nfs-cache-handle-limit\f[R] controls the maximum number 7941 of cached file handles stored by the \f[C]nfsmount\f[R] caching handler. 7942 This should not be set too low or you may experience errors when trying 7943 to access files. 7944 The default is 1000000, but consider lowering this limit if the 7945 server\[aq]s system resource usage causes problems. 7946 .SS macFUSE Notes 7947 .PP 7948 If installing macFUSE using dmg 7949 packages (https://github.com/osxfuse/osxfuse/releases) from the website, 7950 rclone will locate the macFUSE libraries without any further 7951 intervention. 7952 If however, macFUSE is installed using the 7953 macports (https://www.macports.org/) package manager, the following 7954 addition steps are required. 7955 .IP 7956 .nf 7957 \f[C] 7958 sudo mkdir /usr/local/lib 7959 cd /usr/local/lib 7960 sudo ln -s /opt/local/lib/libfuse.2.dylib 7961 \f[R] 7962 .fi 7963 .SS FUSE-T Limitations, Caveats, and Notes 7964 .PP 7965 There are some limitations, caveats, and notes about how it works. 7966 These are current as of FUSE-T version 1.0.14. 7967 .SS ModTime update on read 7968 .PP 7969 As per the FUSE-T 7970 wiki (https://github.com/macos-fuse-t/fuse-t/wiki#caveats): 7971 .RS 7972 .PP 7973 File access and modification times cannot be set separately as it seems 7974 to be an issue with the NFS client which always modifies both. 7975 Can be reproduced with \[aq]touch -m\[aq] and \[aq]touch -a\[aq] 7976 commands 7977 .RE 7978 .PP 7979 This means that viewing files with various tools, notably macOS Finder, 7980 will cause rlcone to update the modification time of the file. 7981 This may make rclone upload a full new copy of the file. 7982 .SS Read Only mounts 7983 .PP 7984 When mounting with \f[C]--read-only\f[R], attempts to write to files 7985 will fail \f[I]silently\f[R] as opposed to with a clear warning as in 7986 macFUSE. 7987 .SS Limitations 7988 .PP 7989 Without the use of \f[C]--vfs-cache-mode\f[R] this can only write files 7990 sequentially, it can only seek when reading. 7991 This means that many applications won\[aq]t work with their files on an 7992 rclone mount without \f[C]--vfs-cache-mode writes\f[R] or 7993 \f[C]--vfs-cache-mode full\f[R]. 7994 See the VFS File Caching section for more info. 7995 When using NFS mount on macOS, if you don\[aq]t specify 7996 |--vfs-cache-mode| the mount point will be read-only. 7997 .PP 7998 The bucket-based remotes (e.g. 7999 Swift, S3, Google Compute Storage, B2) do not support the concept of 8000 empty directories, so empty directories will have a tendency to 8001 disappear once they fall out of the directory cache. 8002 .PP 8003 When \f[C]rclone mount\f[R] is invoked on Unix with \f[C]--daemon\f[R] 8004 flag, the main rclone program will wait for the background mount to 8005 become ready or until the timeout specified by the 8006 \f[C]--daemon-wait\f[R] flag. 8007 On Linux it can check mount status using ProcFS so the flag in fact sets 8008 \f[B]maximum\f[R] time to wait, while the real wait can be less. 8009 On macOS / BSD the time to wait is constant and the check is performed 8010 only at the end. 8011 We advise you to set wait time on macOS reasonably. 8012 .PP 8013 Only supported on Linux, FreeBSD, OS X and Windows at the moment. 8014 .SS rclone nfsmount vs rclone sync/copy 8015 .PP 8016 File systems expect things to be 100% reliable, whereas cloud storage 8017 systems are a long way from 100% reliable. 8018 The rclone sync/copy commands cope with this with lots of retries. 8019 However rclone nfsmount can\[aq]t use retries in the same way without 8020 making local copies of the uploads. 8021 Look at the VFS File Caching for solutions to make nfsmount more 8022 reliable. 8023 .SS Attribute caching 8024 .PP 8025 You can use the flag \f[C]--attr-timeout\f[R] to set the time the kernel 8026 caches the attributes (size, modification time, etc.) for directory 8027 entries. 8028 .PP 8029 The default is \f[C]1s\f[R] which caches files just long enough to avoid 8030 too many callbacks to rclone from the kernel. 8031 .PP 8032 In theory 0s should be the correct value for filesystems which can 8033 change outside the control of the kernel. 8034 However this causes quite a few problems such as rclone using too much 8035 memory (https://github.com/artpar/artpar/issues/2157), rclone not 8036 serving files to 8037 samba (https://forum.rclone.org/t/rclone-1-39-vs-1-40-mount-issue/5112) 8038 and excessive time listing 8039 directories (https://github.com/artpar/artpar/issues/2095#issuecomment-371141147). 8040 .PP 8041 The kernel can cache the info about a file for the time given by 8042 \f[C]--attr-timeout\f[R]. 8043 You may see corruption if the remote file changes length during this 8044 window. 8045 It will show up as either a truncated file or a file with garbage on the 8046 end. 8047 With \f[C]--attr-timeout 1s\f[R] this is very unlikely but not 8048 impossible. 8049 The higher you set \f[C]--attr-timeout\f[R] the more likely it is. 8050 The default setting of \[dq]1s\[dq] is the lowest setting which 8051 mitigates the problems above. 8052 .PP 8053 If you set it higher (\f[C]10s\f[R] or \f[C]1m\f[R] say) then the kernel 8054 will call back to rclone less often making it more efficient, however 8055 there is more chance of the corruption issue above. 8056 .PP 8057 If files don\[aq]t change on the remote outside of the control of rclone 8058 then there is no chance of corruption. 8059 .PP 8060 This is the same as setting the attr_timeout option in mount.fuse. 8061 .SS Filters 8062 .PP 8063 Note that all the rclone filters can be used to select a subset of the 8064 files to be visible in the mount. 8065 .SS systemd 8066 .PP 8067 When running rclone nfsmount as a systemd service, it is possible to use 8068 Type=notify. 8069 In this case the service will enter the started state after the 8070 mountpoint has been successfully set up. 8071 Units having the rclone nfsmount service specified as a requirement will 8072 see all files and folders immediately in this mode. 8073 .PP 8074 Note that systemd runs mount units without any environment variables 8075 including \f[C]PATH\f[R] or \f[C]HOME\f[R]. 8076 This means that tilde (\f[C]\[ti]\f[R]) expansion will not work and you 8077 should provide \f[C]--config\f[R] and \f[C]--cache-dir\f[R] explicitly 8078 as absolute paths via rclone arguments. 8079 Since mounting requires the \f[C]fusermount\f[R] program, rclone will 8080 use the fallback PATH of \f[C]/bin:/usr/bin\f[R] in this scenario. 8081 Please ensure that \f[C]fusermount\f[R] is present on this PATH. 8082 .SS Rclone as Unix mount helper 8083 .PP 8084 The core Unix program \f[C]/bin/mount\f[R] normally takes the 8085 \f[C]-t FSTYPE\f[R] argument then runs the \f[C]/sbin/mount.FSTYPE\f[R] 8086 helper program passing it mount options as \f[C]-o key=val,...\f[R] or 8087 \f[C]--opt=...\f[R]. 8088 Automount (classic or systemd) behaves in a similar way. 8089 .PP 8090 rclone by default expects GNU-style flags \f[C]--key val\f[R]. 8091 To run it as a mount helper you should symlink rclone binary to 8092 \f[C]/sbin/mount.rclone\f[R] and optionally \f[C]/usr/bin/rclonefs\f[R], 8093 e.g. 8094 \f[C]ln -s /usr/bin/rclone /sbin/mount.rclone\f[R]. 8095 rclone will detect it and translate command-line arguments 8096 appropriately. 8097 .PP 8098 Now you can run classic mounts like this: 8099 .IP 8100 .nf 8101 \f[C] 8102 mount sftp1:subdir /mnt/data -t rclone -o vfs_cache_mode=writes,sftp_key_file=/path/to/pem 8103 \f[R] 8104 .fi 8105 .PP 8106 or create systemd mount units: 8107 .IP 8108 .nf 8109 \f[C] 8110 # /etc/systemd/system/mnt-data.mount 8111 [Unit] 8112 Description=Mount for /mnt/data 8113 [Mount] 8114 Type=rclone 8115 What=sftp1:subdir 8116 Where=/mnt/data 8117 Options=rw,_netdev,allow_other,args2env,vfs-cache-mode=writes,config=/etc/rclone.conf,cache-dir=/var/rclone 8118 \f[R] 8119 .fi 8120 .PP 8121 optionally accompanied by systemd automount unit 8122 .IP 8123 .nf 8124 \f[C] 8125 # /etc/systemd/system/mnt-data.automount 8126 [Unit] 8127 Description=AutoMount for /mnt/data 8128 [Automount] 8129 Where=/mnt/data 8130 TimeoutIdleSec=600 8131 [Install] 8132 WantedBy=multi-user.target 8133 \f[R] 8134 .fi 8135 .PP 8136 or add in \f[C]/etc/fstab\f[R] a line like 8137 .IP 8138 .nf 8139 \f[C] 8140 sftp1:subdir /mnt/data rclone rw,noauto,nofail,_netdev,x-systemd.automount,args2env,vfs_cache_mode=writes,config=/etc/rclone.conf,cache_dir=/var/cache/rclone 0 0 8141 \f[R] 8142 .fi 8143 .PP 8144 or use classic Automountd. 8145 Remember to provide explicit \f[C]config=...,cache-dir=...\f[R] as a 8146 workaround for mount units being run without \f[C]HOME\f[R]. 8147 .PP 8148 Rclone in the mount helper mode will split \f[C]-o\f[R] argument(s) by 8149 comma, replace \f[C]_\f[R] by \f[C]-\f[R] and prepend \f[C]--\f[R] to 8150 get the command-line flags. 8151 Options containing commas or spaces can be wrapped in single or double 8152 quotes. 8153 Any inner quotes inside outer quotes of the same type should be doubled. 8154 .PP 8155 Mount option syntax includes a few extra options treated specially: 8156 .IP \[bu] 2 8157 \f[C]env.NAME=VALUE\f[R] will set an environment variable for the mount 8158 process. 8159 This helps with Automountd and Systemd.mount which don\[aq]t allow 8160 setting custom environment for mount helpers. 8161 Typically you will use \f[C]env.HTTPS_PROXY=proxy.host:3128\f[R] or 8162 \f[C]env.HOME=/root\f[R] 8163 .IP \[bu] 2 8164 \f[C]command=cmount\f[R] can be used to run \f[C]cmount\f[R] or any 8165 other rclone command rather than the default \f[C]mount\f[R]. 8166 .IP \[bu] 2 8167 \f[C]args2env\f[R] will pass mount options to the mount helper running 8168 in background via environment variables instead of command line 8169 arguments. 8170 This allows to hide secrets from such commands as \f[C]ps\f[R] or 8171 \f[C]pgrep\f[R]. 8172 .IP \[bu] 2 8173 \f[C]vv...\f[R] will be transformed into appropriate 8174 \f[C]--verbose=N\f[R] 8175 .IP \[bu] 2 8176 standard mount options like \f[C]x-systemd.automount\f[R], 8177 \f[C]_netdev\f[R], \f[C]nosuid\f[R] and alike are intended only for 8178 Automountd and ignored by rclone. 8179 ## VFS - Virtual File System 8180 .PP 8181 This command uses the VFS layer. 8182 This adapts the cloud storage objects that rclone uses into something 8183 which looks much more like a disk filing system. 8184 .PP 8185 Cloud storage objects have lots of properties which aren\[aq]t like disk 8186 files - you can\[aq]t extend them or write to the middle of them, so the 8187 VFS layer has to deal with that. 8188 Because there is no one right way of doing this there are various 8189 options explained below. 8190 .PP 8191 The VFS layer also implements a directory cache - this caches info about 8192 files and directories (but not the data) in memory. 8193 .SS VFS Directory Cache 8194 .PP 8195 Using the \f[C]--dir-cache-time\f[R] flag, you can control how long a 8196 directory should be considered up to date and not refreshed from the 8197 backend. 8198 Changes made through the VFS will appear immediately or invalidate the 8199 cache. 8200 .IP 8201 .nf 8202 \f[C] 8203 --dir-cache-time duration Time to cache directory entries for (default 5m0s) 8204 --poll-interval duration Time to wait between polling for changes. Must be smaller than dir-cache-time. Only on supported remotes. Set to 0 to disable (default 1m0s) 8205 \f[R] 8206 .fi 8207 .PP 8208 However, changes made directly on the cloud storage by the web interface 8209 or a different copy of rclone will only be picked up once the directory 8210 cache expires if the backend configured does not support polling for 8211 changes. 8212 If the backend supports polling, changes will be picked up within the 8213 polling interval. 8214 .PP 8215 You can send a \f[C]SIGHUP\f[R] signal to rclone for it to flush all 8216 directory caches, regardless of how old they are. 8217 Assuming only one rclone instance is running, you can reset the cache 8218 like this: 8219 .IP 8220 .nf 8221 \f[C] 8222 kill -SIGHUP $(pidof rclone) 8223 \f[R] 8224 .fi 8225 .PP 8226 If you configure rclone with a remote control then you can use rclone rc 8227 to flush the whole directory cache: 8228 .IP 8229 .nf 8230 \f[C] 8231 rclone rc vfs/forget 8232 \f[R] 8233 .fi 8234 .PP 8235 Or individual files or directories: 8236 .IP 8237 .nf 8238 \f[C] 8239 rclone rc vfs/forget file=path/to/file dir=path/to/dir 8240 \f[R] 8241 .fi 8242 .SS VFS File Buffering 8243 .PP 8244 The \f[C]--buffer-size\f[R] flag determines the amount of memory, that 8245 will be used to buffer data in advance. 8246 .PP 8247 Each open file will try to keep the specified amount of data in memory 8248 at all times. 8249 The buffered data is bound to one open file and won\[aq]t be shared. 8250 .PP 8251 This flag is a upper limit for the used memory per open file. 8252 The buffer will only use memory for data that is downloaded but not not 8253 yet read. 8254 If the buffer is empty, only a small amount of memory will be used. 8255 .PP 8256 The maximum memory used by rclone for buffering can be up to 8257 \f[C]--buffer-size * open files\f[R]. 8258 .SS VFS File Caching 8259 .PP 8260 These flags control the VFS file caching options. 8261 File caching is necessary to make the VFS layer appear compatible with a 8262 normal file system. 8263 It can be disabled at the cost of some compatibility. 8264 .PP 8265 For example you\[aq]ll need to enable VFS caching if you want to read 8266 and write simultaneously to a file. 8267 See below for more details. 8268 .PP 8269 Note that the VFS cache is separate from the cache backend and you may 8270 find that you need one or the other or both. 8271 .IP 8272 .nf 8273 \f[C] 8274 --cache-dir string Directory rclone will use for caching. 8275 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off) 8276 --vfs-cache-max-age duration Max time since last access of objects in the cache (default 1h0m0s) 8277 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache (default off) 8278 --vfs-cache-min-free-space SizeSuffix Target minimum free space on the disk containing the cache (default off) 8279 --vfs-cache-poll-interval duration Interval to poll the cache for stale objects (default 1m0s) 8280 --vfs-write-back duration Time to writeback files after last use when using cache (default 5s) 8281 \f[R] 8282 .fi 8283 .PP 8284 If run with \f[C]-vv\f[R] rclone will print the location of the file 8285 cache. 8286 The files are stored in the user cache file area which is OS dependent 8287 but can be controlled with \f[C]--cache-dir\f[R] or setting the 8288 appropriate environment variable. 8289 .PP 8290 The cache has 4 different modes selected by \f[C]--vfs-cache-mode\f[R]. 8291 The higher the cache mode the more compatible rclone becomes at the cost 8292 of using disk space. 8293 .PP 8294 Note that files are written back to the remote only when they are closed 8295 and if they haven\[aq]t been accessed for \f[C]--vfs-write-back\f[R] 8296 seconds. 8297 If rclone is quit or dies with files that haven\[aq]t been uploaded, 8298 these will be uploaded next time rclone is run with the same flags. 8299 .PP 8300 If using \f[C]--vfs-cache-max-size\f[R] or 8301 \f[C]--vfs-cache-min-free-size\f[R] note that the cache may exceed these 8302 quotas for two reasons. 8303 Firstly because it is only checked every 8304 \f[C]--vfs-cache-poll-interval\f[R]. 8305 Secondly because open files cannot be evicted from the cache. 8306 When \f[C]--vfs-cache-max-size\f[R] or 8307 \f[C]--vfs-cache-min-free-size\f[R] is exceeded, rclone will attempt to 8308 evict the least accessed files from the cache first. 8309 rclone will start with files that haven\[aq]t been accessed for the 8310 longest. 8311 This cache flushing strategy is efficient and more relevant files are 8312 likely to remain cached. 8313 .PP 8314 The \f[C]--vfs-cache-max-age\f[R] will evict files from the cache after 8315 the set time since last access has passed. 8316 The default value of 1 hour will start evicting files from cache that 8317 haven\[aq]t been accessed for 1 hour. 8318 When a cached file is accessed the 1 hour timer is reset to 0 and will 8319 wait for 1 more hour before evicting. 8320 Specify the time with standard notation, s, m, h, d, w . 8321 .PP 8322 You \f[B]should not\f[R] run two copies of rclone using the same VFS 8323 cache with the same or overlapping remotes if using 8324 \f[C]--vfs-cache-mode > off\f[R]. 8325 This can potentially cause data corruption if you do. 8326 You can work around this by giving each rclone its own cache hierarchy 8327 with \f[C]--cache-dir\f[R]. 8328 You don\[aq]t need to worry about this if the remotes in use don\[aq]t 8329 overlap. 8330 .SS --vfs-cache-mode off 8331 .PP 8332 In this mode (the default) the cache will read directly from the remote 8333 and write directly to the remote without caching anything on disk. 8334 .PP 8335 This will mean some operations are not possible 8336 .IP \[bu] 2 8337 Files can\[aq]t be opened for both read AND write 8338 .IP \[bu] 2 8339 Files opened for write can\[aq]t be seeked 8340 .IP \[bu] 2 8341 Existing files opened for write must have O_TRUNC set 8342 .IP \[bu] 2 8343 Files open for read with O_TRUNC will be opened write only 8344 .IP \[bu] 2 8345 Files open for write only will behave as if O_TRUNC was supplied 8346 .IP \[bu] 2 8347 Open modes O_APPEND, O_TRUNC are ignored 8348 .IP \[bu] 2 8349 If an upload fails it can\[aq]t be retried 8350 .SS --vfs-cache-mode minimal 8351 .PP 8352 This is very similar to \[dq]off\[dq] except that files opened for read 8353 AND write will be buffered to disk. 8354 This means that files opened for write will be a lot more compatible, 8355 but uses the minimal disk space. 8356 .PP 8357 These operations are not possible 8358 .IP \[bu] 2 8359 Files opened for write only can\[aq]t be seeked 8360 .IP \[bu] 2 8361 Existing files opened for write must have O_TRUNC set 8362 .IP \[bu] 2 8363 Files opened for write only will ignore O_APPEND, O_TRUNC 8364 .IP \[bu] 2 8365 If an upload fails it can\[aq]t be retried 8366 .SS --vfs-cache-mode writes 8367 .PP 8368 In this mode files opened for read only are still read directly from the 8369 remote, write only and read/write files are buffered to disk first. 8370 .PP 8371 This mode should support all normal file system operations. 8372 .PP 8373 If an upload fails it will be retried at exponentially increasing 8374 intervals up to 1 minute. 8375 .SS --vfs-cache-mode full 8376 .PP 8377 In this mode all reads and writes are buffered to and from disk. 8378 When data is read from the remote this is buffered to disk as well. 8379 .PP 8380 In this mode the files in the cache will be sparse files and rclone will 8381 keep track of which bits of the files it has downloaded. 8382 .PP 8383 So if an application only reads the starts of each file, then rclone 8384 will only buffer the start of the file. 8385 These files will appear to be their full size in the cache, but they 8386 will be sparse files with only the data that has been downloaded present 8387 in them. 8388 .PP 8389 This mode should support all normal file system operations and is 8390 otherwise identical to \f[C]--vfs-cache-mode\f[R] writes. 8391 .PP 8392 When reading a file rclone will read \f[C]--buffer-size\f[R] plus 8393 \f[C]--vfs-read-ahead\f[R] bytes ahead. 8394 The \f[C]--buffer-size\f[R] is buffered in memory whereas the 8395 \f[C]--vfs-read-ahead\f[R] is buffered on disk. 8396 .PP 8397 When using this mode it is recommended that \f[C]--buffer-size\f[R] is 8398 not set too large and \f[C]--vfs-read-ahead\f[R] is set large if 8399 required. 8400 .PP 8401 \f[B]IMPORTANT\f[R] not all file systems support sparse files. 8402 In particular FAT/exFAT do not. 8403 Rclone will perform very badly if the cache directory is on a filesystem 8404 which doesn\[aq]t support sparse files and it will log an ERROR message 8405 if one is detected. 8406 .SS Fingerprinting 8407 .PP 8408 Various parts of the VFS use fingerprinting to see if a local file copy 8409 has changed relative to a remote file. 8410 Fingerprints are made from: 8411 .IP \[bu] 2 8412 size 8413 .IP \[bu] 2 8414 modification time 8415 .IP \[bu] 2 8416 hash 8417 .PP 8418 where available on an object. 8419 .PP 8420 On some backends some of these attributes are slow to read (they take an 8421 extra API call per object, or extra work per object). 8422 .PP 8423 For example \f[C]hash\f[R] is slow with the \f[C]local\f[R] and 8424 \f[C]sftp\f[R] backends as they have to read the entire file and hash 8425 it, and \f[C]modtime\f[R] is slow with the \f[C]s3\f[R], 8426 \f[C]swift\f[R], \f[C]ftp\f[R] and \f[C]qinqstor\f[R] backends because 8427 they need to do an extra API call to fetch it. 8428 .PP 8429 If you use the \f[C]--vfs-fast-fingerprint\f[R] flag then rclone will 8430 not include the slow operations in the fingerprint. 8431 This makes the fingerprinting less accurate but much faster and will 8432 improve the opening time of cached files. 8433 .PP 8434 If you are running a vfs cache over \f[C]local\f[R], \f[C]s3\f[R] or 8435 \f[C]swift\f[R] backends then using this flag is recommended. 8436 .PP 8437 Note that if you change the value of this flag, the fingerprints of the 8438 files in the cache may be invalidated and the files will need to be 8439 downloaded again. 8440 .SS VFS Chunked Reading 8441 .PP 8442 When rclone reads files from a remote it reads them in chunks. 8443 This means that rather than requesting the whole file rclone reads the 8444 chunk specified. 8445 This can reduce the used download quota for some remotes by requesting 8446 only chunks from the remote that are actually read, at the cost of an 8447 increased number of requests. 8448 .PP 8449 These flags control the chunking: 8450 .IP 8451 .nf 8452 \f[C] 8453 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks (default 128M) 8454 --vfs-read-chunk-size-limit SizeSuffix Max chunk doubling size (default off) 8455 \f[R] 8456 .fi 8457 .PP 8458 Rclone will start reading a chunk of size 8459 \f[C]--vfs-read-chunk-size\f[R], and then double the size for each read. 8460 When \f[C]--vfs-read-chunk-size-limit\f[R] is specified, and greater 8461 than \f[C]--vfs-read-chunk-size\f[R], the chunk size for each open file 8462 will get doubled only until the specified value is reached. 8463 If the value is \[dq]off\[dq], which is the default, the limit is 8464 disabled and the chunk size will grow indefinitely. 8465 .PP 8466 With \f[C]--vfs-read-chunk-size 100M\f[R] and 8467 \f[C]--vfs-read-chunk-size-limit 0\f[R] the following parts will be 8468 downloaded: 0-100M, 100M-200M, 200M-300M, 300M-400M and so on. 8469 When \f[C]--vfs-read-chunk-size-limit 500M\f[R] is specified, the result 8470 would be 0-100M, 100M-300M, 300M-700M, 700M-1200M, 1200M-1700M and so 8471 on. 8472 .PP 8473 Setting \f[C]--vfs-read-chunk-size\f[R] to \f[C]0\f[R] or \[dq]off\[dq] 8474 disables chunked reading. 8475 .SS VFS Performance 8476 .PP 8477 These flags may be used to enable/disable features of the VFS for 8478 performance or other reasons. 8479 See also the chunked reading feature. 8480 .PP 8481 In particular S3 and Swift benefit hugely from the 8482 \f[C]--no-modtime\f[R] flag (or use \f[C]--use-server-modtime\f[R] for a 8483 slightly different effect) as each read of the modification time takes a 8484 transaction. 8485 .IP 8486 .nf 8487 \f[C] 8488 --no-checksum Don\[aq]t compare checksums on up/download. 8489 --no-modtime Don\[aq]t read/write the modification time (can speed things up). 8490 --no-seek Don\[aq]t allow seeking in files. 8491 --read-only Only allow read-only access. 8492 \f[R] 8493 .fi 8494 .PP 8495 Sometimes rclone is delivered reads or writes out of order. 8496 Rather than seeking rclone will wait a short time for the in sequence 8497 read or write to come in. 8498 These flags only come into effect when not using an on disk cache file. 8499 .IP 8500 .nf 8501 \f[C] 8502 --vfs-read-wait duration Time to wait for in-sequence read before seeking (default 20ms) 8503 --vfs-write-wait duration Time to wait for in-sequence write before giving error (default 1s) 8504 \f[R] 8505 .fi 8506 .PP 8507 When using VFS write caching (\f[C]--vfs-cache-mode\f[R] with value 8508 writes or full), the global flag \f[C]--transfers\f[R] can be set to 8509 adjust the number of parallel uploads of modified files from the cache 8510 (the related global flag \f[C]--checkers\f[R] has no effect on the VFS). 8511 .IP 8512 .nf 8513 \f[C] 8514 --transfers int Number of file transfers to run in parallel (default 4) 8515 \f[R] 8516 .fi 8517 .SS VFS Case Sensitivity 8518 .PP 8519 Linux file systems are case-sensitive: two files can differ only by 8520 case, and the exact case must be used when opening a file. 8521 .PP 8522 File systems in modern Windows are case-insensitive but case-preserving: 8523 although existing files can be opened using any case, the exact case 8524 used to create the file is preserved and available for programs to 8525 query. 8526 It is not allowed for two files in the same directory to differ only by 8527 case. 8528 .PP 8529 Usually file systems on macOS are case-insensitive. 8530 It is possible to make macOS file systems case-sensitive but that is not 8531 the default. 8532 .PP 8533 The \f[C]--vfs-case-insensitive\f[R] VFS flag controls how rclone 8534 handles these two cases. 8535 If its value is \[dq]false\[dq], rclone passes file names to the remote 8536 as-is. 8537 If the flag is \[dq]true\[dq] (or appears without a value on the command 8538 line), rclone may perform a \[dq]fixup\[dq] as explained below. 8539 .PP 8540 The user may specify a file name to open/delete/rename/etc with a case 8541 different than what is stored on the remote. 8542 If an argument refers to an existing file with exactly the same name, 8543 then the case of the existing file on the disk will be used. 8544 However, if a file name with exactly the same name is not found but a 8545 name differing only by case exists, rclone will transparently fixup the 8546 name. 8547 This fixup happens only when an existing file is requested. 8548 Case sensitivity of file names created anew by rclone is controlled by 8549 the underlying remote. 8550 .PP 8551 Note that case sensitivity of the operating system running rclone (the 8552 target) may differ from case sensitivity of a file system presented by 8553 rclone (the source). 8554 The flag controls whether \[dq]fixup\[dq] is performed to satisfy the 8555 target. 8556 .PP 8557 If the flag is not provided on the command line, then its default value 8558 depends on the operating system where rclone runs: \[dq]true\[dq] on 8559 Windows and macOS, \[dq]false\[dq] otherwise. 8560 If the flag is provided without a value, then it is \[dq]true\[dq]. 8561 .PP 8562 The \f[C]--no-unicode-normalization\f[R] flag controls whether a similar 8563 \[dq]fixup\[dq] is performed for filenames that differ but are 8564 canonically 8565 equivalent (https://en.wikipedia.org/wiki/Unicode_equivalence) with 8566 respect to unicode. 8567 Unicode normalization can be particularly helpful for users of macOS, 8568 which prefers form NFD instead of the NFC used by most other platforms. 8569 It is therefore highly recommended to keep the default of 8570 \f[C]false\f[R] on macOS, to avoid encoding compatibility issues. 8571 .PP 8572 In the (probably unlikely) event that a directory has multiple duplicate 8573 filenames after applying case and unicode normalization, the 8574 \f[C]--vfs-block-norm-dupes\f[R] flag allows hiding these duplicates. 8575 This comes with a performance tradeoff, as rclone will have to scan the 8576 entire directory for duplicates when listing a directory. 8577 For this reason, it is recommended to leave this disabled if not needed. 8578 However, macOS users may wish to consider using it, as otherwise, if a 8579 remote directory contains both NFC and NFD versions of the same 8580 filename, an odd situation will occur: both versions of the file will be 8581 visible in the mount, and both will appear to be editable, however, 8582 editing either version will actually result in only the NFD version 8583 getting edited under the hood. 8584 \f[C]--vfs-block- norm-dupes\f[R] prevents this confusion by detecting 8585 this scenario, hiding the duplicates, and logging an error, similar to 8586 how this is handled in \f[C]rclone sync\f[R]. 8587 .SS VFS Disk Options 8588 .PP 8589 This flag allows you to manually set the statistics about the filing 8590 system. 8591 It can be useful when those statistics cannot be read correctly 8592 automatically. 8593 .IP 8594 .nf 8595 \f[C] 8596 --vfs-disk-space-total-size Manually set the total disk space size (example: 256G, default: -1) 8597 \f[R] 8598 .fi 8599 .SS Alternate report of used bytes 8600 .PP 8601 Some backends, most notably S3, do not report the amount of bytes used. 8602 If you need this information to be available when running \f[C]df\f[R] 8603 on the filesystem, then pass the flag \f[C]--vfs-used-is-size\f[R] to 8604 rclone. 8605 With this flag set, instead of relying on the backend to report this 8606 information, rclone will scan the whole remote similar to 8607 \f[C]rclone size\f[R] and compute the total used space itself. 8608 .PP 8609 \f[I]WARNING.\f[R] Contrary to \f[C]rclone size\f[R], this flag ignores 8610 filters so that the result is accurate. 8611 However, this is very inefficient and may cost lots of API calls 8612 resulting in extra charges. 8613 Use it as a last resort and only with caching. 8614 .IP 8615 .nf 8616 \f[C] 8617 rclone nfsmount remote:path /path/to/mountpoint [flags] 8618 \f[R] 8619 .fi 8620 .SS Options 8621 .IP 8622 .nf 8623 \f[C] 8624 --addr string IPaddress:Port or :Port to bind server to 8625 --allow-non-empty Allow mounting over a non-empty directory (not supported on Windows) 8626 --allow-other Allow access to other users (not supported on Windows) 8627 --allow-root Allow access to root user (not supported on Windows) 8628 --async-read Use asynchronous reads (not supported on Windows) (default true) 8629 --attr-timeout Duration Time for which file/directory attributes are cached (default 1s) 8630 --daemon Run mount in background and exit parent process (as background output is suppressed, use --log-file with --log-format=pid,... to monitor) (not supported on Windows) 8631 --daemon-timeout Duration Time limit for rclone to respond to kernel (not supported on Windows) (default 0s) 8632 --daemon-wait Duration Time to wait for ready mount from daemon (maximum time on Linux, constant sleep time on OSX/BSD) (not supported on Windows) (default 1m0s) 8633 --debug-fuse Debug the FUSE internals - needs -v 8634 --default-permissions Makes kernel enforce access control based on the file mode (not supported on Windows) 8635 --devname string Set the device name - default is remote:path 8636 --dir-cache-time Duration Time to cache directory entries for (default 5m0s) 8637 --dir-perms FileMode Directory permissions (default 0777) 8638 --file-perms FileMode File permissions (default 0666) 8639 --fuse-flag stringArray Flags or arguments to be passed direct to libfuse/WinFsp (repeat if required) 8640 --gid uint32 Override the gid field set by the filesystem (not supported on Windows) (default 1000) 8641 -h, --help help for nfsmount 8642 --max-read-ahead SizeSuffix The number of bytes that can be prefetched for sequential reads (not supported on Windows) (default 128Ki) 8643 --mount-case-insensitive Tristate Tell the OS the mount is case insensitive (true) or sensitive (false) regardless of the backend (auto) (default unset) 8644 --network-mode Mount as remote network drive, instead of fixed disk drive (supported on Windows only) 8645 --nfs-cache-handle-limit int max file handles cached simultaneously (min 5) (default 1000000) 8646 --no-checksum Don\[aq]t compare checksums on up/download 8647 --no-modtime Don\[aq]t read/write the modification time (can speed things up) 8648 --no-seek Don\[aq]t allow seeking in files 8649 --noappledouble Ignore Apple Double (._) and .DS_Store files (supported on OSX only) (default true) 8650 --noapplexattr Ignore all \[dq]com.apple.*\[dq] extended attributes (supported on OSX only) 8651 -o, --option stringArray Option for libfuse/WinFsp (repeat if required) 8652 --poll-interval Duration Time to wait between polling for changes, must be smaller than dir-cache-time and only on supported remotes (set 0 to disable) (default 1m0s) 8653 --read-only Only allow read-only access 8654 --sudo Use sudo to run the mount command as root. 8655 --uid uint32 Override the uid field set by the filesystem (not supported on Windows) (default 1000) 8656 --umask int Override the permission bits set by the filesystem (not supported on Windows) (default 2) 8657 --vfs-block-norm-dupes If duplicate filenames exist in the same directory (after normalization), log an error and hide the duplicates (may have a performance cost) 8658 --vfs-cache-max-age Duration Max time since last access of objects in the cache (default 1h0m0s) 8659 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache (default off) 8660 --vfs-cache-min-free-space SizeSuffix Target minimum free space on the disk containing the cache (default off) 8661 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off) 8662 --vfs-cache-poll-interval Duration Interval to poll the cache for stale objects (default 1m0s) 8663 --vfs-case-insensitive If a file name not found, find a case insensitive match 8664 --vfs-disk-space-total-size SizeSuffix Specify the total space of disk (default off) 8665 --vfs-fast-fingerprint Use fast (less accurate) fingerprints for change detection 8666 --vfs-read-ahead SizeSuffix Extra read ahead over --buffer-size when using cache-mode full 8667 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks (default 128Mi) 8668 --vfs-read-chunk-size-limit SizeSuffix If greater than --vfs-read-chunk-size, double the chunk size after each chunk read, until the limit is reached (\[aq]off\[aq] is unlimited) (default off) 8669 --vfs-read-wait Duration Time to wait for in-sequence read before seeking (default 20ms) 8670 --vfs-refresh Refreshes the directory cache recursively in the background on start 8671 --vfs-used-is-size rclone size Use the rclone size algorithm for Used size 8672 --vfs-write-back Duration Time to writeback files after last use when using cache (default 5s) 8673 --vfs-write-wait Duration Time to wait for in-sequence write before giving error (default 1s) 8674 --volname string Set the volume name (supported on Windows and OSX only) 8675 --write-back-cache Makes kernel buffer writes before sending them to rclone (without this, writethrough caching is used) (not supported on Windows) 8676 \f[R] 8677 .fi 8678 .SS Filter Options 8679 .PP 8680 Flags for filtering directory listings. 8681 .IP 8682 .nf 8683 \f[C] 8684 --delete-excluded Delete files on dest excluded from sync 8685 --exclude stringArray Exclude files matching pattern 8686 --exclude-from stringArray Read file exclude patterns from file (use - to read from stdin) 8687 --exclude-if-present stringArray Exclude directories if filename is present 8688 --files-from stringArray Read list of source-file names from file (use - to read from stdin) 8689 --files-from-raw stringArray Read list of source-file names from file without any processing of lines (use - to read from stdin) 8690 -f, --filter stringArray Add a file filtering rule 8691 --filter-from stringArray Read file filtering patterns from a file (use - to read from stdin) 8692 --ignore-case Ignore case in filters (case insensitive) 8693 --include stringArray Include files matching pattern 8694 --include-from stringArray Read file include patterns from file (use - to read from stdin) 8695 --max-age Duration Only transfer files younger than this in s or suffix ms|s|m|h|d|w|M|y (default off) 8696 --max-depth int If set limits the recursion depth to this (default -1) 8697 --max-size SizeSuffix Only transfer files smaller than this in KiB or suffix B|K|M|G|T|P (default off) 8698 --metadata-exclude stringArray Exclude metadatas matching pattern 8699 --metadata-exclude-from stringArray Read metadata exclude patterns from file (use - to read from stdin) 8700 --metadata-filter stringArray Add a metadata filtering rule 8701 --metadata-filter-from stringArray Read metadata filtering patterns from a file (use - to read from stdin) 8702 --metadata-include stringArray Include metadatas matching pattern 8703 --metadata-include-from stringArray Read metadata include patterns from file (use - to read from stdin) 8704 --min-age Duration Only transfer files older than this in s or suffix ms|s|m|h|d|w|M|y (default off) 8705 --min-size SizeSuffix Only transfer files bigger than this in KiB or suffix B|K|M|G|T|P (default off) 8706 \f[R] 8707 .fi 8708 .PP 8709 See the global flags page (https://rclone.org/flags/) for global options 8710 not listed here. 8711 .SH SEE ALSO 8712 .IP \[bu] 2 8713 rclone (https://rclone.org/commands/rclone/) - Show help for rclone 8714 commands, flags and backends. 8715 .SH rclone obscure 8716 .PP 8717 Obscure password for use in the rclone config file. 8718 .SS Synopsis 8719 .PP 8720 In the rclone config file, human-readable passwords are obscured. 8721 Obscuring them is done by encrypting them and writing them out in 8722 base64. 8723 This is \f[B]not\f[R] a secure way of encrypting these passwords as 8724 rclone can decrypt them - it is to prevent \[dq]eyedropping\[dq] - 8725 namely someone seeing a password in the rclone config file by accident. 8726 .PP 8727 Many equally important things (like access tokens) are not obscured in 8728 the config file. 8729 However it is very hard to shoulder surf a 64 character hex token. 8730 .PP 8731 This command can also accept a password through STDIN instead of an 8732 argument by passing a hyphen as an argument. 8733 This will use the first line of STDIN as the password not including the 8734 trailing newline. 8735 .IP 8736 .nf 8737 \f[C] 8738 echo \[dq]secretpassword\[dq] | rclone obscure - 8739 \f[R] 8740 .fi 8741 .PP 8742 If there is no data on STDIN to read, rclone obscure will default to 8743 obfuscating the hyphen itself. 8744 .PP 8745 If you want to encrypt the config file then please use config file 8746 encryption - see rclone 8747 config (https://rclone.org/commands/rclone_config/) for more info. 8748 .IP 8749 .nf 8750 \f[C] 8751 rclone obscure password [flags] 8752 \f[R] 8753 .fi 8754 .SS Options 8755 .IP 8756 .nf 8757 \f[C] 8758 -h, --help help for obscure 8759 \f[R] 8760 .fi 8761 .PP 8762 See the global flags page (https://rclone.org/flags/) for global options 8763 not listed here. 8764 .SH SEE ALSO 8765 .IP \[bu] 2 8766 rclone (https://rclone.org/commands/rclone/) - Show help for rclone 8767 commands, flags and backends. 8768 .SH rclone rc 8769 .PP 8770 Run a command against a running rclone. 8771 .SS Synopsis 8772 .PP 8773 This runs a command against a running rclone. 8774 Use the \f[C]--url\f[R] flag to specify an non default URL to connect 8775 on. 8776 This can be either a \[dq]:port\[dq] which is taken to mean 8777 \[dq]http://localhost:port\[dq] or a \[dq]host:port\[dq] which is taken 8778 to mean \[dq]http://host:port\[dq] 8779 .PP 8780 A username and password can be passed in with \f[C]--user\f[R] and 8781 \f[C]--pass\f[R]. 8782 .PP 8783 Note that \f[C]--rc-addr\f[R], \f[C]--rc-user\f[R], \f[C]--rc-pass\f[R] 8784 will be read also for \f[C]--url\f[R], \f[C]--user\f[R], 8785 \f[C]--pass\f[R]. 8786 .PP 8787 Arguments should be passed in as parameter=value. 8788 .PP 8789 The result will be returned as a JSON object by default. 8790 .PP 8791 The \f[C]--json\f[R] parameter can be used to pass in a JSON blob as an 8792 input instead of key=value arguments. 8793 This is the only way of passing in more complicated values. 8794 .PP 8795 The \f[C]-o\f[R]/\f[C]--opt\f[R] option can be used to set a key 8796 \[dq]opt\[dq] with key, value options in the form \f[C]-o key=value\f[R] 8797 or \f[C]-o key\f[R]. 8798 It can be repeated as many times as required. 8799 This is useful for rc commands which take the \[dq]opt\[dq] parameter 8800 which by convention is a dictionary of strings. 8801 .IP 8802 .nf 8803 \f[C] 8804 -o key=value -o key2 8805 \f[R] 8806 .fi 8807 .PP 8808 Will place this in the \[dq]opt\[dq] value 8809 .IP 8810 .nf 8811 \f[C] 8812 {\[dq]key\[dq]:\[dq]value\[dq], \[dq]key2\[dq],\[dq]\[dq]) 8813 \f[R] 8814 .fi 8815 .PP 8816 The \f[C]-a\f[R]/\f[C]--arg\f[R] option can be used to set strings in 8817 the \[dq]arg\[dq] value. 8818 It can be repeated as many times as required. 8819 This is useful for rc commands which take the \[dq]arg\[dq] parameter 8820 which by convention is a list of strings. 8821 .IP 8822 .nf 8823 \f[C] 8824 -a value -a value2 8825 \f[R] 8826 .fi 8827 .PP 8828 Will place this in the \[dq]arg\[dq] value 8829 .IP 8830 .nf 8831 \f[C] 8832 [\[dq]value\[dq], \[dq]value2\[dq]] 8833 \f[R] 8834 .fi 8835 .PP 8836 Use \f[C]--loopback\f[R] to connect to the rclone instance running 8837 \f[C]rclone rc\f[R]. 8838 This is very useful for testing commands without having to run an rclone 8839 rc server, e.g.: 8840 .IP 8841 .nf 8842 \f[C] 8843 rclone rc --loopback operations/about fs=/ 8844 \f[R] 8845 .fi 8846 .PP 8847 Use \f[C]rclone rc\f[R] to see a list of all possible commands. 8848 .IP 8849 .nf 8850 \f[C] 8851 rclone rc commands parameter [flags] 8852 \f[R] 8853 .fi 8854 .SS Options 8855 .IP 8856 .nf 8857 \f[C] 8858 -a, --arg stringArray Argument placed in the \[dq]arg\[dq] array 8859 -h, --help help for rc 8860 --json string Input JSON - use instead of key=value args 8861 --loopback If set connect to this rclone instance not via HTTP 8862 --no-output If set, don\[aq]t output the JSON result 8863 -o, --opt stringArray Option in the form name=value or name placed in the \[dq]opt\[dq] array 8864 --pass string Password to use to connect to rclone remote control 8865 --url string URL to connect to rclone remote control (default \[dq]http://localhost:5572/\[dq]) 8866 --user string Username to use to rclone remote control 8867 \f[R] 8868 .fi 8869 .PP 8870 See the global flags page (https://rclone.org/flags/) for global options 8871 not listed here. 8872 .SH SEE ALSO 8873 .IP \[bu] 2 8874 rclone (https://rclone.org/commands/rclone/) - Show help for rclone 8875 commands, flags and backends. 8876 .SH rclone rcat 8877 .PP 8878 Copies standard input to file on remote. 8879 .SS Synopsis 8880 .PP 8881 rclone rcat reads from standard input (stdin) and copies it to a single 8882 remote file. 8883 .IP 8884 .nf 8885 \f[C] 8886 echo \[dq]hello world\[dq] | rclone rcat remote:path/to/file 8887 ffmpeg - | rclone rcat remote:path/to/file 8888 \f[R] 8889 .fi 8890 .PP 8891 If the remote file already exists, it will be overwritten. 8892 .PP 8893 rcat will try to upload small files in a single request, which is 8894 usually more efficient than the streaming/chunked upload endpoints, 8895 which use multiple requests. 8896 Exact behaviour depends on the remote. 8897 What is considered a small file may be set through 8898 \f[C]--streaming-upload-cutoff\f[R]. 8899 Uploading only starts after the cutoff is reached or if the file ends 8900 before that. 8901 The data must fit into RAM. 8902 The cutoff needs to be small enough to adhere the limits of your remote, 8903 please see there. 8904 Generally speaking, setting this cutoff too high will decrease your 8905 performance. 8906 .PP 8907 Use the \f[C]--size\f[R] flag to preallocate the file in advance at the 8908 remote end and actually stream it, even if remote backend doesn\[aq]t 8909 support streaming. 8910 .PP 8911 \f[C]--size\f[R] should be the exact size of the input stream in bytes. 8912 If the size of the stream is different in length to the \f[C]--size\f[R] 8913 passed in then the transfer will likely fail. 8914 .PP 8915 Note that the upload cannot be retried because the data is not stored. 8916 If the backend supports multipart uploading then individual chunks can 8917 be retried. 8918 If you need to transfer a lot of data, you may be better off caching it 8919 locally and then \f[C]rclone move\f[R] it to the destination which can 8920 use retries. 8921 .IP 8922 .nf 8923 \f[C] 8924 rclone rcat remote:path [flags] 8925 \f[R] 8926 .fi 8927 .SS Options 8928 .IP 8929 .nf 8930 \f[C] 8931 -h, --help help for rcat 8932 --size int File size hint to preallocate (default -1) 8933 \f[R] 8934 .fi 8935 .SS Important Options 8936 .PP 8937 Important flags useful for most commands. 8938 .IP 8939 .nf 8940 \f[C] 8941 -n, --dry-run Do a trial run with no permanent changes 8942 -i, --interactive Enable interactive mode 8943 -v, --verbose count Print lots more stuff (repeat for more) 8944 \f[R] 8945 .fi 8946 .PP 8947 See the global flags page (https://rclone.org/flags/) for global options 8948 not listed here. 8949 .SH SEE ALSO 8950 .IP \[bu] 2 8951 rclone (https://rclone.org/commands/rclone/) - Show help for rclone 8952 commands, flags and backends. 8953 .SH rclone rcd 8954 .PP 8955 Run rclone listening to remote control commands only. 8956 .SS Synopsis 8957 .PP 8958 This runs rclone so that it only listens to remote control commands. 8959 .PP 8960 This is useful if you are controlling rclone via the rc API. 8961 .PP 8962 If you pass in a path to a directory, rclone will serve that directory 8963 for GET requests on the URL passed in. 8964 It will also open the URL in the browser when rclone is run. 8965 .PP 8966 See the rc documentation (https://rclone.org/rc/) for more info on the 8967 rc flags. 8968 .SS Server options 8969 .PP 8970 Use \f[C]--rc-addr\f[R] to specify which IP address and port the server 8971 should listen on, eg \f[C]--rc-addr 1.2.3.4:8000\f[R] or 8972 \f[C]--rc-addr :8080\f[R] to listen to all IPs. 8973 By default it only listens on localhost. 8974 You can use port :0 to let the OS choose an available port. 8975 .PP 8976 If you set \f[C]--rc-addr\f[R] to listen on a public or LAN accessible 8977 IP address then using Authentication is advised - see the next section 8978 for info. 8979 .PP 8980 You can use a unix socket by setting the url to 8981 \f[C]unix:///path/to/socket\f[R] or just by using an absolute path name. 8982 Note that unix sockets bypass the authentication - this is expected to 8983 be done with file system permissions. 8984 .PP 8985 \f[C]--rc-addr\f[R] may be repeated to listen on multiple 8986 IPs/ports/sockets. 8987 .PP 8988 \f[C]--rc-server-read-timeout\f[R] and 8989 \f[C]--rc-server-write-timeout\f[R] can be used to control the timeouts 8990 on the server. 8991 Note that this is the total time for a transfer. 8992 .PP 8993 \f[C]--rc-max-header-bytes\f[R] controls the maximum number of bytes the 8994 server will accept in the HTTP header. 8995 .PP 8996 \f[C]--rc-baseurl\f[R] controls the URL prefix that rclone serves from. 8997 By default rclone will serve from the root. 8998 If you used \f[C]--rc-baseurl \[dq]/rclone\[dq]\f[R] then rclone would 8999 serve from a URL starting with \[dq]/rclone/\[dq]. 9000 This is useful if you wish to proxy rclone serve. 9001 Rclone automatically inserts leading and trailing \[dq]/\[dq] on 9002 \f[C]--rc-baseurl\f[R], so \f[C]--rc-baseurl \[dq]rclone\[dq]\f[R], 9003 \f[C]--rc-baseurl \[dq]/rclone\[dq]\f[R] and 9004 \f[C]--rc-baseurl \[dq]/rclone/\[dq]\f[R] are all treated identically. 9005 .SS TLS (SSL) 9006 .PP 9007 By default this will serve over http. 9008 If you want you can serve over https. 9009 You will need to supply the \f[C]--rc-cert\f[R] and \f[C]--rc-key\f[R] 9010 flags. 9011 If you wish to do client side certificate validation then you will need 9012 to supply \f[C]--rc-client-ca\f[R] also. 9013 .PP 9014 \f[C]--rc-cert\f[R] should be a either a PEM encoded certificate or a 9015 concatenation of that with the CA certificate. 9016 \f[C]--krc-ey\f[R] should be the PEM encoded private key and 9017 \f[C]--rc-client-ca\f[R] should be the PEM encoded client certificate 9018 authority certificate. 9019 .PP 9020 --rc-min-tls-version is minimum TLS version that is acceptable. 9021 Valid values are \[dq]tls1.0\[dq], \[dq]tls1.1\[dq], \[dq]tls1.2\[dq] 9022 and \[dq]tls1.3\[dq] (default \[dq]tls1.0\[dq]). 9023 .SS Template 9024 .PP 9025 \f[C]--rc-template\f[R] allows a user to specify a custom markup 9026 template for HTTP and WebDAV serve functions. 9027 The server exports the following markup to be used within the template 9028 to server pages: 9029 .PP 9030 .TS 9031 tab(@); 9032 lw(35.0n) lw(35.0n). 9033 T{ 9034 Parameter 9035 T}@T{ 9036 Description 9037 T} 9038 _ 9039 T{ 9040 \&.Name 9041 T}@T{ 9042 The full path of a file/directory. 9043 T} 9044 T{ 9045 \&.Title 9046 T}@T{ 9047 Directory listing of .Name 9048 T} 9049 T{ 9050 \&.Sort 9051 T}@T{ 9052 The current sort used. 9053 This is changeable via ?sort= parameter 9054 T} 9055 T{ 9056 T}@T{ 9057 Sort Options: namedirfirst,name,size,time (default namedirfirst) 9058 T} 9059 T{ 9060 \&.Order 9061 T}@T{ 9062 The current ordering used. 9063 This is changeable via ?order= parameter 9064 T} 9065 T{ 9066 T}@T{ 9067 Order Options: asc,desc (default asc) 9068 T} 9069 T{ 9070 \&.Query 9071 T}@T{ 9072 Currently unused. 9073 T} 9074 T{ 9075 \&.Breadcrumb 9076 T}@T{ 9077 Allows for creating a relative navigation 9078 T} 9079 T{ 9080 -- .Link 9081 T}@T{ 9082 The relative to the root link of the Text. 9083 T} 9084 T{ 9085 -- .Text 9086 T}@T{ 9087 The Name of the directory. 9088 T} 9089 T{ 9090 \&.Entries 9091 T}@T{ 9092 Information about a specific file/directory. 9093 T} 9094 T{ 9095 -- .URL 9096 T}@T{ 9097 The \[aq]url\[aq] of an entry. 9098 T} 9099 T{ 9100 -- .Leaf 9101 T}@T{ 9102 Currently same as \[aq]URL\[aq] but intended to be \[aq]just\[aq] the 9103 name. 9104 T} 9105 T{ 9106 -- .IsDir 9107 T}@T{ 9108 Boolean for if an entry is a directory or not. 9109 T} 9110 T{ 9111 -- .Size 9112 T}@T{ 9113 Size in Bytes of the entry. 9114 T} 9115 T{ 9116 -- .ModTime 9117 T}@T{ 9118 The UTC timestamp of an entry. 9119 T} 9120 .TE 9121 .PP 9122 The server also makes the following functions available so that they can 9123 be used within the template. 9124 These functions help extend the options for dynamic rendering of HTML. 9125 They can be used to render HTML based on specific conditions. 9126 .PP 9127 .TS 9128 tab(@); 9129 lw(35.0n) lw(35.0n). 9130 T{ 9131 Function 9132 T}@T{ 9133 Description 9134 T} 9135 _ 9136 T{ 9137 afterEpoch 9138 T}@T{ 9139 Returns the time since the epoch for the given time. 9140 T} 9141 T{ 9142 contains 9143 T}@T{ 9144 Checks whether a given substring is present or not in a given string. 9145 T} 9146 T{ 9147 hasPrefix 9148 T}@T{ 9149 Checks whether the given string begins with the specified prefix. 9150 T} 9151 T{ 9152 hasSuffix 9153 T}@T{ 9154 Checks whether the given string end with the specified suffix. 9155 T} 9156 .TE 9157 .SS Authentication 9158 .PP 9159 By default this will serve files without needing a login. 9160 .PP 9161 You can either use an htpasswd file which can take lots of users, or set 9162 a single username and password with the \f[C]--rc-user\f[R] and 9163 \f[C]--rc-pass\f[R] flags. 9164 .PP 9165 If no static users are configured by either of the above methods, and 9166 client certificates are required by the \f[C]--client-ca\f[R] flag 9167 passed to the server, the client certificate common name will be 9168 considered as the username. 9169 .PP 9170 Use \f[C]--rc-htpasswd /path/to/htpasswd\f[R] to provide an htpasswd 9171 file. 9172 This is in standard apache format and supports MD5, SHA1 and BCrypt for 9173 basic authentication. 9174 Bcrypt is recommended. 9175 .PP 9176 To create an htpasswd file: 9177 .IP 9178 .nf 9179 \f[C] 9180 touch htpasswd 9181 htpasswd -B htpasswd user 9182 htpasswd -B htpasswd anotherUser 9183 \f[R] 9184 .fi 9185 .PP 9186 The password file can be updated while rclone is running. 9187 .PP 9188 Use \f[C]--rc-realm\f[R] to set the authentication realm. 9189 .PP 9190 Use \f[C]--rc-salt\f[R] to change the password hashing salt from the 9191 default. 9192 .IP 9193 .nf 9194 \f[C] 9195 rclone rcd <path to files to serve>* [flags] 9196 \f[R] 9197 .fi 9198 .SS Options 9199 .IP 9200 .nf 9201 \f[C] 9202 -h, --help help for rcd 9203 \f[R] 9204 .fi 9205 .SS RC Options 9206 .PP 9207 Flags to control the Remote Control API. 9208 .IP 9209 .nf 9210 \f[C] 9211 --rc Enable the remote control server 9212 --rc-addr stringArray IPaddress:Port or :Port to bind server to (default [localhost:5572]) 9213 --rc-allow-origin string Origin which cross-domain request (CORS) can be executed from 9214 --rc-baseurl string Prefix for URLs - leave blank for root 9215 --rc-cert string TLS PEM key (concatenation of certificate and CA certificate) 9216 --rc-client-ca string Client certificate authority to verify clients with 9217 --rc-enable-metrics Enable prometheus metrics on /metrics 9218 --rc-files string Path to local files to serve on the HTTP server 9219 --rc-htpasswd string A htpasswd file - if not provided no authentication is done 9220 --rc-job-expire-duration Duration Expire finished async jobs older than this value (default 1m0s) 9221 --rc-job-expire-interval Duration Interval to check for expired async jobs (default 10s) 9222 --rc-key string TLS PEM Private key 9223 --rc-max-header-bytes int Maximum size of request header (default 4096) 9224 --rc-min-tls-version string Minimum TLS version that is acceptable (default \[dq]tls1.0\[dq]) 9225 --rc-no-auth Don\[aq]t require auth for certain methods 9226 --rc-pass string Password for authentication 9227 --rc-realm string Realm for authentication 9228 --rc-salt string Password hashing salt (default \[dq]dlPL2MqE\[dq]) 9229 --rc-serve Enable the serving of remote objects 9230 --rc-server-read-timeout Duration Timeout for server reading data (default 1h0m0s) 9231 --rc-server-write-timeout Duration Timeout for server writing data (default 1h0m0s) 9232 --rc-template string User-specified template 9233 --rc-user string User name for authentication 9234 --rc-web-fetch-url string URL to fetch the releases for webgui (default \[dq]https://api.github.com/repos/artpar/artpar-webui-react/releases/latest\[dq]) 9235 --rc-web-gui Launch WebGUI on localhost 9236 --rc-web-gui-force-update Force update to latest version of web gui 9237 --rc-web-gui-no-open-browser Don\[aq]t open the browser automatically 9238 --rc-web-gui-update Check and update to latest version of web gui 9239 \f[R] 9240 .fi 9241 .PP 9242 See the global flags page (https://rclone.org/flags/) for global options 9243 not listed here. 9244 .SH SEE ALSO 9245 .IP \[bu] 2 9246 rclone (https://rclone.org/commands/rclone/) - Show help for rclone 9247 commands, flags and backends. 9248 .SH rclone rmdirs 9249 .PP 9250 Remove empty directories under the path. 9251 .SS Synopsis 9252 .PP 9253 This recursively removes any empty directories (including directories 9254 that only contain empty directories), that it finds under the path. 9255 The root path itself will also be removed if it is empty, unless you 9256 supply the \f[C]--leave-root\f[R] flag. 9257 .PP 9258 Use command rmdir (https://rclone.org/commands/rclone_rmdir/) to delete 9259 just the empty directory given by path, not recurse. 9260 .PP 9261 This is useful for tidying up remotes that rclone has left a lot of 9262 empty directories in. 9263 For example the delete (https://rclone.org/commands/rclone_delete/) 9264 command will delete files but leave the directory structure (unless used 9265 with option \f[C]--rmdirs\f[R]). 9266 .PP 9267 This will delete \f[C]--checkers\f[R] directories concurrently so if you 9268 have thousands of empty directories consider increasing this number. 9269 .PP 9270 To delete a path and any objects in it, use the 9271 purge (https://rclone.org/commands/rclone_purge/) command. 9272 .IP 9273 .nf 9274 \f[C] 9275 rclone rmdirs remote:path [flags] 9276 \f[R] 9277 .fi 9278 .SS Options 9279 .IP 9280 .nf 9281 \f[C] 9282 -h, --help help for rmdirs 9283 --leave-root Do not remove root directory if empty 9284 \f[R] 9285 .fi 9286 .SS Important Options 9287 .PP 9288 Important flags useful for most commands. 9289 .IP 9290 .nf 9291 \f[C] 9292 -n, --dry-run Do a trial run with no permanent changes 9293 -i, --interactive Enable interactive mode 9294 -v, --verbose count Print lots more stuff (repeat for more) 9295 \f[R] 9296 .fi 9297 .PP 9298 See the global flags page (https://rclone.org/flags/) for global options 9299 not listed here. 9300 .SH SEE ALSO 9301 .IP \[bu] 2 9302 rclone (https://rclone.org/commands/rclone/) - Show help for rclone 9303 commands, flags and backends. 9304 .SH rclone selfupdate 9305 .PP 9306 Update the rclone binary. 9307 .SS Synopsis 9308 .PP 9309 This command downloads the latest release of rclone and replaces the 9310 currently running binary. 9311 The download is verified with a hashsum and cryptographically signed 9312 signature; see the release signing 9313 docs (https://rclone.org/release_signing/) for details. 9314 .PP 9315 If used without flags (or with implied \f[C]--stable\f[R] flag), this 9316 command will install the latest stable release. 9317 However, some issues may be fixed (or features added) only in the latest 9318 beta release. 9319 In such cases you should run the command with the \f[C]--beta\f[R] flag, 9320 i.e. 9321 \f[C]rclone selfupdate --beta\f[R]. 9322 You can check in advance what version would be installed by adding the 9323 \f[C]--check\f[R] flag, then repeat the command without it when you are 9324 satisfied. 9325 .PP 9326 Sometimes the rclone team may recommend you a concrete beta or stable 9327 rclone release to troubleshoot your issue or add a bleeding edge 9328 feature. 9329 The \f[C]--version VER\f[R] flag, if given, will update to the concrete 9330 version instead of the latest one. 9331 If you omit micro version from \f[C]VER\f[R] (for example 9332 \f[C]1.53\f[R]), the latest matching micro version will be used. 9333 .PP 9334 Upon successful update rclone will print a message that contains a 9335 previous version number. 9336 You will need it if you later decide to revert your update for some 9337 reason. 9338 Then you\[aq]ll have to note the previous version and run the following 9339 command: \f[C]rclone selfupdate [--beta] OLDVER\f[R]. 9340 If the old version contains only dots and digits (for example 9341 \f[C]v1.54.0\f[R]) then it\[aq]s a stable release so you won\[aq]t need 9342 the \f[C]--beta\f[R] flag. 9343 Beta releases have an additional information similar to 9344 \f[C]v1.54.0-beta.5111.06f1c0c61\f[R]. 9345 (if you are a developer and use a locally built rclone, the version 9346 number will end with \f[C]-DEV\f[R], you will have to rebuild it as it 9347 obviously can\[aq]t be distributed). 9348 .PP 9349 If you previously installed rclone via a package manager, the package 9350 may include local documentation or configure services. 9351 You may wish to update with the flag \f[C]--package deb\f[R] or 9352 \f[C]--package rpm\f[R] (whichever is correct for your OS) to update 9353 these too. 9354 This command with the default \f[C]--package zip\f[R] will update only 9355 the rclone executable so the local manual may become inaccurate after 9356 it. 9357 .PP 9358 The rclone mount (https://rclone.org/commands/rclone_mount/) command may 9359 or may not support extended FUSE options depending on the build and OS. 9360 \f[C]selfupdate\f[R] will refuse to update if the capability would be 9361 discarded. 9362 .PP 9363 Note: Windows forbids deletion of a currently running executable so this 9364 command will rename the old executable to \[aq]rclone.old.exe\[aq] upon 9365 success. 9366 .PP 9367 Please note that this command was not available before rclone version 9368 1.55. 9369 If it fails for you with the message 9370 \f[C]unknown command \[dq]selfupdate\[dq]\f[R] then you will need to 9371 update manually following the install instructions located at 9372 https://rclone.org/install/ 9373 .IP 9374 .nf 9375 \f[C] 9376 rclone selfupdate [flags] 9377 \f[R] 9378 .fi 9379 .SS Options 9380 .IP 9381 .nf 9382 \f[C] 9383 --beta Install beta release 9384 --check Check for latest release, do not download 9385 -h, --help help for selfupdate 9386 --output string Save the downloaded binary at a given path (default: replace running binary) 9387 --package string Package format: zip|deb|rpm (default: zip) 9388 --stable Install stable release (this is the default) 9389 --version string Install the given rclone version (default: latest) 9390 \f[R] 9391 .fi 9392 .PP 9393 See the global flags page (https://rclone.org/flags/) for global options 9394 not listed here. 9395 .SH SEE ALSO 9396 .IP \[bu] 2 9397 rclone (https://rclone.org/commands/rclone/) - Show help for rclone 9398 commands, flags and backends. 9399 .SH rclone serve 9400 .PP 9401 Serve a remote over a protocol. 9402 .SS Synopsis 9403 .PP 9404 Serve a remote over a given protocol. 9405 Requires the use of a subcommand to specify the protocol, e.g. 9406 .IP 9407 .nf 9408 \f[C] 9409 rclone serve http remote: 9410 \f[R] 9411 .fi 9412 .PP 9413 Each subcommand has its own options which you can see in their help. 9414 .IP 9415 .nf 9416 \f[C] 9417 rclone serve <protocol> [opts] <remote> [flags] 9418 \f[R] 9419 .fi 9420 .SS Options 9421 .IP 9422 .nf 9423 \f[C] 9424 -h, --help help for serve 9425 \f[R] 9426 .fi 9427 .PP 9428 See the global flags page (https://rclone.org/flags/) for global options 9429 not listed here. 9430 .SH SEE ALSO 9431 .IP \[bu] 2 9432 rclone (https://rclone.org/commands/rclone/) - Show help for rclone 9433 commands, flags and backends. 9434 .IP \[bu] 2 9435 rclone serve dlna (https://rclone.org/commands/rclone_serve_dlna/) - 9436 Serve remote:path over DLNA 9437 .IP \[bu] 2 9438 rclone serve docker (https://rclone.org/commands/rclone_serve_docker/) - 9439 Serve any remote on docker\[aq]s volume plugin API. 9440 .IP \[bu] 2 9441 rclone serve ftp (https://rclone.org/commands/rclone_serve_ftp/) - Serve 9442 remote:path over FTP. 9443 .IP \[bu] 2 9444 rclone serve http (https://rclone.org/commands/rclone_serve_http/) - 9445 Serve the remote over HTTP. 9446 .IP \[bu] 2 9447 rclone serve nfs (https://rclone.org/commands/rclone_serve_nfs/) - Serve 9448 the remote as an NFS mount 9449 .IP \[bu] 2 9450 rclone serve restic (https://rclone.org/commands/rclone_serve_restic/) - 9451 Serve the remote for restic\[aq]s REST API. 9452 .IP \[bu] 2 9453 rclone serve s3 (https://rclone.org/commands/rclone_serve_s3/) - Serve 9454 remote:path over s3. 9455 .IP \[bu] 2 9456 rclone serve sftp (https://rclone.org/commands/rclone_serve_sftp/) - 9457 Serve the remote over SFTP. 9458 .IP \[bu] 2 9459 rclone serve webdav (https://rclone.org/commands/rclone_serve_webdav/) - 9460 Serve remote:path over WebDAV. 9461 .SH rclone serve dlna 9462 .PP 9463 Serve remote:path over DLNA 9464 .SS Synopsis 9465 .PP 9466 Run a DLNA media server for media stored in an rclone remote. 9467 Many devices, such as the Xbox and PlayStation, can automatically 9468 discover this server in the LAN and play audio/video from it. 9469 VLC is also supported. 9470 Service discovery uses UDP multicast packets (SSDP) and will thus only 9471 work on LANs. 9472 .PP 9473 Rclone will list all files present in the remote, without filtering 9474 based on media formats or file extensions. 9475 Additionally, there is no media transcoding support. 9476 This means that some players might show files that they are not able to 9477 play back correctly. 9478 .SS Server options 9479 .PP 9480 Use \f[C]--addr\f[R] to specify which IP address and port the server 9481 should listen on, e.g. 9482 \f[C]--addr 1.2.3.4:8000\f[R] or \f[C]--addr :8080\f[R] to listen to all 9483 IPs. 9484 .PP 9485 Use \f[C]--name\f[R] to choose the friendly server name, which is by 9486 default \[dq]rclone (hostname)\[dq]. 9487 .PP 9488 Use \f[C]--log-trace\f[R] in conjunction with \f[C]-vv\f[R] to enable 9489 additional debug logging of all UPNP traffic. 9490 ## VFS - Virtual File System 9491 .PP 9492 This command uses the VFS layer. 9493 This adapts the cloud storage objects that rclone uses into something 9494 which looks much more like a disk filing system. 9495 .PP 9496 Cloud storage objects have lots of properties which aren\[aq]t like disk 9497 files - you can\[aq]t extend them or write to the middle of them, so the 9498 VFS layer has to deal with that. 9499 Because there is no one right way of doing this there are various 9500 options explained below. 9501 .PP 9502 The VFS layer also implements a directory cache - this caches info about 9503 files and directories (but not the data) in memory. 9504 .SS VFS Directory Cache 9505 .PP 9506 Using the \f[C]--dir-cache-time\f[R] flag, you can control how long a 9507 directory should be considered up to date and not refreshed from the 9508 backend. 9509 Changes made through the VFS will appear immediately or invalidate the 9510 cache. 9511 .IP 9512 .nf 9513 \f[C] 9514 --dir-cache-time duration Time to cache directory entries for (default 5m0s) 9515 --poll-interval duration Time to wait between polling for changes. Must be smaller than dir-cache-time. Only on supported remotes. Set to 0 to disable (default 1m0s) 9516 \f[R] 9517 .fi 9518 .PP 9519 However, changes made directly on the cloud storage by the web interface 9520 or a different copy of rclone will only be picked up once the directory 9521 cache expires if the backend configured does not support polling for 9522 changes. 9523 If the backend supports polling, changes will be picked up within the 9524 polling interval. 9525 .PP 9526 You can send a \f[C]SIGHUP\f[R] signal to rclone for it to flush all 9527 directory caches, regardless of how old they are. 9528 Assuming only one rclone instance is running, you can reset the cache 9529 like this: 9530 .IP 9531 .nf 9532 \f[C] 9533 kill -SIGHUP $(pidof rclone) 9534 \f[R] 9535 .fi 9536 .PP 9537 If you configure rclone with a remote control then you can use rclone rc 9538 to flush the whole directory cache: 9539 .IP 9540 .nf 9541 \f[C] 9542 rclone rc vfs/forget 9543 \f[R] 9544 .fi 9545 .PP 9546 Or individual files or directories: 9547 .IP 9548 .nf 9549 \f[C] 9550 rclone rc vfs/forget file=path/to/file dir=path/to/dir 9551 \f[R] 9552 .fi 9553 .SS VFS File Buffering 9554 .PP 9555 The \f[C]--buffer-size\f[R] flag determines the amount of memory, that 9556 will be used to buffer data in advance. 9557 .PP 9558 Each open file will try to keep the specified amount of data in memory 9559 at all times. 9560 The buffered data is bound to one open file and won\[aq]t be shared. 9561 .PP 9562 This flag is a upper limit for the used memory per open file. 9563 The buffer will only use memory for data that is downloaded but not not 9564 yet read. 9565 If the buffer is empty, only a small amount of memory will be used. 9566 .PP 9567 The maximum memory used by rclone for buffering can be up to 9568 \f[C]--buffer-size * open files\f[R]. 9569 .SS VFS File Caching 9570 .PP 9571 These flags control the VFS file caching options. 9572 File caching is necessary to make the VFS layer appear compatible with a 9573 normal file system. 9574 It can be disabled at the cost of some compatibility. 9575 .PP 9576 For example you\[aq]ll need to enable VFS caching if you want to read 9577 and write simultaneously to a file. 9578 See below for more details. 9579 .PP 9580 Note that the VFS cache is separate from the cache backend and you may 9581 find that you need one or the other or both. 9582 .IP 9583 .nf 9584 \f[C] 9585 --cache-dir string Directory rclone will use for caching. 9586 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off) 9587 --vfs-cache-max-age duration Max time since last access of objects in the cache (default 1h0m0s) 9588 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache (default off) 9589 --vfs-cache-min-free-space SizeSuffix Target minimum free space on the disk containing the cache (default off) 9590 --vfs-cache-poll-interval duration Interval to poll the cache for stale objects (default 1m0s) 9591 --vfs-write-back duration Time to writeback files after last use when using cache (default 5s) 9592 \f[R] 9593 .fi 9594 .PP 9595 If run with \f[C]-vv\f[R] rclone will print the location of the file 9596 cache. 9597 The files are stored in the user cache file area which is OS dependent 9598 but can be controlled with \f[C]--cache-dir\f[R] or setting the 9599 appropriate environment variable. 9600 .PP 9601 The cache has 4 different modes selected by \f[C]--vfs-cache-mode\f[R]. 9602 The higher the cache mode the more compatible rclone becomes at the cost 9603 of using disk space. 9604 .PP 9605 Note that files are written back to the remote only when they are closed 9606 and if they haven\[aq]t been accessed for \f[C]--vfs-write-back\f[R] 9607 seconds. 9608 If rclone is quit or dies with files that haven\[aq]t been uploaded, 9609 these will be uploaded next time rclone is run with the same flags. 9610 .PP 9611 If using \f[C]--vfs-cache-max-size\f[R] or 9612 \f[C]--vfs-cache-min-free-size\f[R] note that the cache may exceed these 9613 quotas for two reasons. 9614 Firstly because it is only checked every 9615 \f[C]--vfs-cache-poll-interval\f[R]. 9616 Secondly because open files cannot be evicted from the cache. 9617 When \f[C]--vfs-cache-max-size\f[R] or 9618 \f[C]--vfs-cache-min-free-size\f[R] is exceeded, rclone will attempt to 9619 evict the least accessed files from the cache first. 9620 rclone will start with files that haven\[aq]t been accessed for the 9621 longest. 9622 This cache flushing strategy is efficient and more relevant files are 9623 likely to remain cached. 9624 .PP 9625 The \f[C]--vfs-cache-max-age\f[R] will evict files from the cache after 9626 the set time since last access has passed. 9627 The default value of 1 hour will start evicting files from cache that 9628 haven\[aq]t been accessed for 1 hour. 9629 When a cached file is accessed the 1 hour timer is reset to 0 and will 9630 wait for 1 more hour before evicting. 9631 Specify the time with standard notation, s, m, h, d, w . 9632 .PP 9633 You \f[B]should not\f[R] run two copies of rclone using the same VFS 9634 cache with the same or overlapping remotes if using 9635 \f[C]--vfs-cache-mode > off\f[R]. 9636 This can potentially cause data corruption if you do. 9637 You can work around this by giving each rclone its own cache hierarchy 9638 with \f[C]--cache-dir\f[R]. 9639 You don\[aq]t need to worry about this if the remotes in use don\[aq]t 9640 overlap. 9641 .SS --vfs-cache-mode off 9642 .PP 9643 In this mode (the default) the cache will read directly from the remote 9644 and write directly to the remote without caching anything on disk. 9645 .PP 9646 This will mean some operations are not possible 9647 .IP \[bu] 2 9648 Files can\[aq]t be opened for both read AND write 9649 .IP \[bu] 2 9650 Files opened for write can\[aq]t be seeked 9651 .IP \[bu] 2 9652 Existing files opened for write must have O_TRUNC set 9653 .IP \[bu] 2 9654 Files open for read with O_TRUNC will be opened write only 9655 .IP \[bu] 2 9656 Files open for write only will behave as if O_TRUNC was supplied 9657 .IP \[bu] 2 9658 Open modes O_APPEND, O_TRUNC are ignored 9659 .IP \[bu] 2 9660 If an upload fails it can\[aq]t be retried 9661 .SS --vfs-cache-mode minimal 9662 .PP 9663 This is very similar to \[dq]off\[dq] except that files opened for read 9664 AND write will be buffered to disk. 9665 This means that files opened for write will be a lot more compatible, 9666 but uses the minimal disk space. 9667 .PP 9668 These operations are not possible 9669 .IP \[bu] 2 9670 Files opened for write only can\[aq]t be seeked 9671 .IP \[bu] 2 9672 Existing files opened for write must have O_TRUNC set 9673 .IP \[bu] 2 9674 Files opened for write only will ignore O_APPEND, O_TRUNC 9675 .IP \[bu] 2 9676 If an upload fails it can\[aq]t be retried 9677 .SS --vfs-cache-mode writes 9678 .PP 9679 In this mode files opened for read only are still read directly from the 9680 remote, write only and read/write files are buffered to disk first. 9681 .PP 9682 This mode should support all normal file system operations. 9683 .PP 9684 If an upload fails it will be retried at exponentially increasing 9685 intervals up to 1 minute. 9686 .SS --vfs-cache-mode full 9687 .PP 9688 In this mode all reads and writes are buffered to and from disk. 9689 When data is read from the remote this is buffered to disk as well. 9690 .PP 9691 In this mode the files in the cache will be sparse files and rclone will 9692 keep track of which bits of the files it has downloaded. 9693 .PP 9694 So if an application only reads the starts of each file, then rclone 9695 will only buffer the start of the file. 9696 These files will appear to be their full size in the cache, but they 9697 will be sparse files with only the data that has been downloaded present 9698 in them. 9699 .PP 9700 This mode should support all normal file system operations and is 9701 otherwise identical to \f[C]--vfs-cache-mode\f[R] writes. 9702 .PP 9703 When reading a file rclone will read \f[C]--buffer-size\f[R] plus 9704 \f[C]--vfs-read-ahead\f[R] bytes ahead. 9705 The \f[C]--buffer-size\f[R] is buffered in memory whereas the 9706 \f[C]--vfs-read-ahead\f[R] is buffered on disk. 9707 .PP 9708 When using this mode it is recommended that \f[C]--buffer-size\f[R] is 9709 not set too large and \f[C]--vfs-read-ahead\f[R] is set large if 9710 required. 9711 .PP 9712 \f[B]IMPORTANT\f[R] not all file systems support sparse files. 9713 In particular FAT/exFAT do not. 9714 Rclone will perform very badly if the cache directory is on a filesystem 9715 which doesn\[aq]t support sparse files and it will log an ERROR message 9716 if one is detected. 9717 .SS Fingerprinting 9718 .PP 9719 Various parts of the VFS use fingerprinting to see if a local file copy 9720 has changed relative to a remote file. 9721 Fingerprints are made from: 9722 .IP \[bu] 2 9723 size 9724 .IP \[bu] 2 9725 modification time 9726 .IP \[bu] 2 9727 hash 9728 .PP 9729 where available on an object. 9730 .PP 9731 On some backends some of these attributes are slow to read (they take an 9732 extra API call per object, or extra work per object). 9733 .PP 9734 For example \f[C]hash\f[R] is slow with the \f[C]local\f[R] and 9735 \f[C]sftp\f[R] backends as they have to read the entire file and hash 9736 it, and \f[C]modtime\f[R] is slow with the \f[C]s3\f[R], 9737 \f[C]swift\f[R], \f[C]ftp\f[R] and \f[C]qinqstor\f[R] backends because 9738 they need to do an extra API call to fetch it. 9739 .PP 9740 If you use the \f[C]--vfs-fast-fingerprint\f[R] flag then rclone will 9741 not include the slow operations in the fingerprint. 9742 This makes the fingerprinting less accurate but much faster and will 9743 improve the opening time of cached files. 9744 .PP 9745 If you are running a vfs cache over \f[C]local\f[R], \f[C]s3\f[R] or 9746 \f[C]swift\f[R] backends then using this flag is recommended. 9747 .PP 9748 Note that if you change the value of this flag, the fingerprints of the 9749 files in the cache may be invalidated and the files will need to be 9750 downloaded again. 9751 .SS VFS Chunked Reading 9752 .PP 9753 When rclone reads files from a remote it reads them in chunks. 9754 This means that rather than requesting the whole file rclone reads the 9755 chunk specified. 9756 This can reduce the used download quota for some remotes by requesting 9757 only chunks from the remote that are actually read, at the cost of an 9758 increased number of requests. 9759 .PP 9760 These flags control the chunking: 9761 .IP 9762 .nf 9763 \f[C] 9764 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks (default 128M) 9765 --vfs-read-chunk-size-limit SizeSuffix Max chunk doubling size (default off) 9766 \f[R] 9767 .fi 9768 .PP 9769 Rclone will start reading a chunk of size 9770 \f[C]--vfs-read-chunk-size\f[R], and then double the size for each read. 9771 When \f[C]--vfs-read-chunk-size-limit\f[R] is specified, and greater 9772 than \f[C]--vfs-read-chunk-size\f[R], the chunk size for each open file 9773 will get doubled only until the specified value is reached. 9774 If the value is \[dq]off\[dq], which is the default, the limit is 9775 disabled and the chunk size will grow indefinitely. 9776 .PP 9777 With \f[C]--vfs-read-chunk-size 100M\f[R] and 9778 \f[C]--vfs-read-chunk-size-limit 0\f[R] the following parts will be 9779 downloaded: 0-100M, 100M-200M, 200M-300M, 300M-400M and so on. 9780 When \f[C]--vfs-read-chunk-size-limit 500M\f[R] is specified, the result 9781 would be 0-100M, 100M-300M, 300M-700M, 700M-1200M, 1200M-1700M and so 9782 on. 9783 .PP 9784 Setting \f[C]--vfs-read-chunk-size\f[R] to \f[C]0\f[R] or \[dq]off\[dq] 9785 disables chunked reading. 9786 .SS VFS Performance 9787 .PP 9788 These flags may be used to enable/disable features of the VFS for 9789 performance or other reasons. 9790 See also the chunked reading feature. 9791 .PP 9792 In particular S3 and Swift benefit hugely from the 9793 \f[C]--no-modtime\f[R] flag (or use \f[C]--use-server-modtime\f[R] for a 9794 slightly different effect) as each read of the modification time takes a 9795 transaction. 9796 .IP 9797 .nf 9798 \f[C] 9799 --no-checksum Don\[aq]t compare checksums on up/download. 9800 --no-modtime Don\[aq]t read/write the modification time (can speed things up). 9801 --no-seek Don\[aq]t allow seeking in files. 9802 --read-only Only allow read-only access. 9803 \f[R] 9804 .fi 9805 .PP 9806 Sometimes rclone is delivered reads or writes out of order. 9807 Rather than seeking rclone will wait a short time for the in sequence 9808 read or write to come in. 9809 These flags only come into effect when not using an on disk cache file. 9810 .IP 9811 .nf 9812 \f[C] 9813 --vfs-read-wait duration Time to wait for in-sequence read before seeking (default 20ms) 9814 --vfs-write-wait duration Time to wait for in-sequence write before giving error (default 1s) 9815 \f[R] 9816 .fi 9817 .PP 9818 When using VFS write caching (\f[C]--vfs-cache-mode\f[R] with value 9819 writes or full), the global flag \f[C]--transfers\f[R] can be set to 9820 adjust the number of parallel uploads of modified files from the cache 9821 (the related global flag \f[C]--checkers\f[R] has no effect on the VFS). 9822 .IP 9823 .nf 9824 \f[C] 9825 --transfers int Number of file transfers to run in parallel (default 4) 9826 \f[R] 9827 .fi 9828 .SS VFS Case Sensitivity 9829 .PP 9830 Linux file systems are case-sensitive: two files can differ only by 9831 case, and the exact case must be used when opening a file. 9832 .PP 9833 File systems in modern Windows are case-insensitive but case-preserving: 9834 although existing files can be opened using any case, the exact case 9835 used to create the file is preserved and available for programs to 9836 query. 9837 It is not allowed for two files in the same directory to differ only by 9838 case. 9839 .PP 9840 Usually file systems on macOS are case-insensitive. 9841 It is possible to make macOS file systems case-sensitive but that is not 9842 the default. 9843 .PP 9844 The \f[C]--vfs-case-insensitive\f[R] VFS flag controls how rclone 9845 handles these two cases. 9846 If its value is \[dq]false\[dq], rclone passes file names to the remote 9847 as-is. 9848 If the flag is \[dq]true\[dq] (or appears without a value on the command 9849 line), rclone may perform a \[dq]fixup\[dq] as explained below. 9850 .PP 9851 The user may specify a file name to open/delete/rename/etc with a case 9852 different than what is stored on the remote. 9853 If an argument refers to an existing file with exactly the same name, 9854 then the case of the existing file on the disk will be used. 9855 However, if a file name with exactly the same name is not found but a 9856 name differing only by case exists, rclone will transparently fixup the 9857 name. 9858 This fixup happens only when an existing file is requested. 9859 Case sensitivity of file names created anew by rclone is controlled by 9860 the underlying remote. 9861 .PP 9862 Note that case sensitivity of the operating system running rclone (the 9863 target) may differ from case sensitivity of a file system presented by 9864 rclone (the source). 9865 The flag controls whether \[dq]fixup\[dq] is performed to satisfy the 9866 target. 9867 .PP 9868 If the flag is not provided on the command line, then its default value 9869 depends on the operating system where rclone runs: \[dq]true\[dq] on 9870 Windows and macOS, \[dq]false\[dq] otherwise. 9871 If the flag is provided without a value, then it is \[dq]true\[dq]. 9872 .PP 9873 The \f[C]--no-unicode-normalization\f[R] flag controls whether a similar 9874 \[dq]fixup\[dq] is performed for filenames that differ but are 9875 canonically 9876 equivalent (https://en.wikipedia.org/wiki/Unicode_equivalence) with 9877 respect to unicode. 9878 Unicode normalization can be particularly helpful for users of macOS, 9879 which prefers form NFD instead of the NFC used by most other platforms. 9880 It is therefore highly recommended to keep the default of 9881 \f[C]false\f[R] on macOS, to avoid encoding compatibility issues. 9882 .PP 9883 In the (probably unlikely) event that a directory has multiple duplicate 9884 filenames after applying case and unicode normalization, the 9885 \f[C]--vfs-block-norm-dupes\f[R] flag allows hiding these duplicates. 9886 This comes with a performance tradeoff, as rclone will have to scan the 9887 entire directory for duplicates when listing a directory. 9888 For this reason, it is recommended to leave this disabled if not needed. 9889 However, macOS users may wish to consider using it, as otherwise, if a 9890 remote directory contains both NFC and NFD versions of the same 9891 filename, an odd situation will occur: both versions of the file will be 9892 visible in the mount, and both will appear to be editable, however, 9893 editing either version will actually result in only the NFD version 9894 getting edited under the hood. 9895 \f[C]--vfs-block- norm-dupes\f[R] prevents this confusion by detecting 9896 this scenario, hiding the duplicates, and logging an error, similar to 9897 how this is handled in \f[C]rclone sync\f[R]. 9898 .SS VFS Disk Options 9899 .PP 9900 This flag allows you to manually set the statistics about the filing 9901 system. 9902 It can be useful when those statistics cannot be read correctly 9903 automatically. 9904 .IP 9905 .nf 9906 \f[C] 9907 --vfs-disk-space-total-size Manually set the total disk space size (example: 256G, default: -1) 9908 \f[R] 9909 .fi 9910 .SS Alternate report of used bytes 9911 .PP 9912 Some backends, most notably S3, do not report the amount of bytes used. 9913 If you need this information to be available when running \f[C]df\f[R] 9914 on the filesystem, then pass the flag \f[C]--vfs-used-is-size\f[R] to 9915 rclone. 9916 With this flag set, instead of relying on the backend to report this 9917 information, rclone will scan the whole remote similar to 9918 \f[C]rclone size\f[R] and compute the total used space itself. 9919 .PP 9920 \f[I]WARNING.\f[R] Contrary to \f[C]rclone size\f[R], this flag ignores 9921 filters so that the result is accurate. 9922 However, this is very inefficient and may cost lots of API calls 9923 resulting in extra charges. 9924 Use it as a last resort and only with caching. 9925 .IP 9926 .nf 9927 \f[C] 9928 rclone serve dlna remote:path [flags] 9929 \f[R] 9930 .fi 9931 .SS Options 9932 .IP 9933 .nf 9934 \f[C] 9935 --addr string The ip:port or :port to bind the DLNA http server to (default \[dq]:7879\[dq]) 9936 --announce-interval Duration The interval between SSDP announcements (default 12m0s) 9937 --dir-cache-time Duration Time to cache directory entries for (default 5m0s) 9938 --dir-perms FileMode Directory permissions (default 0777) 9939 --file-perms FileMode File permissions (default 0666) 9940 --gid uint32 Override the gid field set by the filesystem (not supported on Windows) (default 1000) 9941 -h, --help help for dlna 9942 --interface stringArray The interface to use for SSDP (repeat as necessary) 9943 --log-trace Enable trace logging of SOAP traffic 9944 --name string Name of DLNA server 9945 --no-checksum Don\[aq]t compare checksums on up/download 9946 --no-modtime Don\[aq]t read/write the modification time (can speed things up) 9947 --no-seek Don\[aq]t allow seeking in files 9948 --poll-interval Duration Time to wait between polling for changes, must be smaller than dir-cache-time and only on supported remotes (set 0 to disable) (default 1m0s) 9949 --read-only Only allow read-only access 9950 --uid uint32 Override the uid field set by the filesystem (not supported on Windows) (default 1000) 9951 --umask int Override the permission bits set by the filesystem (not supported on Windows) (default 2) 9952 --vfs-block-norm-dupes If duplicate filenames exist in the same directory (after normalization), log an error and hide the duplicates (may have a performance cost) 9953 --vfs-cache-max-age Duration Max time since last access of objects in the cache (default 1h0m0s) 9954 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache (default off) 9955 --vfs-cache-min-free-space SizeSuffix Target minimum free space on the disk containing the cache (default off) 9956 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off) 9957 --vfs-cache-poll-interval Duration Interval to poll the cache for stale objects (default 1m0s) 9958 --vfs-case-insensitive If a file name not found, find a case insensitive match 9959 --vfs-disk-space-total-size SizeSuffix Specify the total space of disk (default off) 9960 --vfs-fast-fingerprint Use fast (less accurate) fingerprints for change detection 9961 --vfs-read-ahead SizeSuffix Extra read ahead over --buffer-size when using cache-mode full 9962 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks (default 128Mi) 9963 --vfs-read-chunk-size-limit SizeSuffix If greater than --vfs-read-chunk-size, double the chunk size after each chunk read, until the limit is reached (\[aq]off\[aq] is unlimited) (default off) 9964 --vfs-read-wait Duration Time to wait for in-sequence read before seeking (default 20ms) 9965 --vfs-refresh Refreshes the directory cache recursively in the background on start 9966 --vfs-used-is-size rclone size Use the rclone size algorithm for Used size 9967 --vfs-write-back Duration Time to writeback files after last use when using cache (default 5s) 9968 --vfs-write-wait Duration Time to wait for in-sequence write before giving error (default 1s) 9969 \f[R] 9970 .fi 9971 .SS Filter Options 9972 .PP 9973 Flags for filtering directory listings. 9974 .IP 9975 .nf 9976 \f[C] 9977 --delete-excluded Delete files on dest excluded from sync 9978 --exclude stringArray Exclude files matching pattern 9979 --exclude-from stringArray Read file exclude patterns from file (use - to read from stdin) 9980 --exclude-if-present stringArray Exclude directories if filename is present 9981 --files-from stringArray Read list of source-file names from file (use - to read from stdin) 9982 --files-from-raw stringArray Read list of source-file names from file without any processing of lines (use - to read from stdin) 9983 -f, --filter stringArray Add a file filtering rule 9984 --filter-from stringArray Read file filtering patterns from a file (use - to read from stdin) 9985 --ignore-case Ignore case in filters (case insensitive) 9986 --include stringArray Include files matching pattern 9987 --include-from stringArray Read file include patterns from file (use - to read from stdin) 9988 --max-age Duration Only transfer files younger than this in s or suffix ms|s|m|h|d|w|M|y (default off) 9989 --max-depth int If set limits the recursion depth to this (default -1) 9990 --max-size SizeSuffix Only transfer files smaller than this in KiB or suffix B|K|M|G|T|P (default off) 9991 --metadata-exclude stringArray Exclude metadatas matching pattern 9992 --metadata-exclude-from stringArray Read metadata exclude patterns from file (use - to read from stdin) 9993 --metadata-filter stringArray Add a metadata filtering rule 9994 --metadata-filter-from stringArray Read metadata filtering patterns from a file (use - to read from stdin) 9995 --metadata-include stringArray Include metadatas matching pattern 9996 --metadata-include-from stringArray Read metadata include patterns from file (use - to read from stdin) 9997 --min-age Duration Only transfer files older than this in s or suffix ms|s|m|h|d|w|M|y (default off) 9998 --min-size SizeSuffix Only transfer files bigger than this in KiB or suffix B|K|M|G|T|P (default off) 9999 \f[R] 10000 .fi 10001 .PP 10002 See the global flags page (https://rclone.org/flags/) for global options 10003 not listed here. 10004 .SH SEE ALSO 10005 .IP \[bu] 2 10006 rclone serve (https://rclone.org/commands/rclone_serve/) - Serve a 10007 remote over a protocol. 10008 .SH rclone serve docker 10009 .PP 10010 Serve any remote on docker\[aq]s volume plugin API. 10011 .SS Synopsis 10012 .PP 10013 This command implements the Docker volume plugin API allowing docker to 10014 use rclone as a data storage mechanism for various cloud providers. 10015 rclone provides docker volume plugin based on it. 10016 .PP 10017 To create a docker plugin, one must create a Unix or TCP socket that 10018 Docker will look for when you use the plugin and then it listens for 10019 commands from docker daemon and runs the corresponding code when 10020 necessary. 10021 Docker plugins can run as a managed plugin under control of the docker 10022 daemon or as an independent native service. 10023 For testing, you can just run it directly from the command line, for 10024 example: 10025 .IP 10026 .nf 10027 \f[C] 10028 sudo rclone serve docker --base-dir /tmp/rclone-volumes --socket-addr localhost:8787 -vv 10029 \f[R] 10030 .fi 10031 .PP 10032 Running \f[C]rclone serve docker\f[R] will create the said socket, 10033 listening for commands from Docker to create the necessary Volumes. 10034 Normally you need not give the \f[C]--socket-addr\f[R] flag. 10035 The API will listen on the unix domain socket at 10036 \f[C]/run/docker/plugins/rclone.sock\f[R]. 10037 In the example above rclone will create a TCP socket and a small file 10038 \f[C]/etc/docker/plugins/rclone.spec\f[R] containing the socket address. 10039 We use \f[C]sudo\f[R] because both paths are writeable only by the root 10040 user. 10041 .PP 10042 If you later decide to change listening socket, the docker daemon must 10043 be restarted to reconnect to \f[C]/run/docker/plugins/rclone.sock\f[R] 10044 or parse new \f[C]/etc/docker/plugins/rclone.spec\f[R]. 10045 Until you restart, any volume related docker commands will timeout 10046 trying to access the old socket. 10047 Running directly is supported on \f[B]Linux only\f[R], not on Windows or 10048 MacOS. 10049 This is not a problem with managed plugin mode described in details in 10050 the full documentation (https://rclone.org/docker). 10051 .PP 10052 The command will create volume mounts under the path given by 10053 \f[C]--base-dir\f[R] (by default 10054 \f[C]/var/lib/docker-volumes/rclone\f[R] available only to root) and 10055 maintain the JSON formatted file \f[C]docker-plugin.state\f[R] in the 10056 rclone cache directory with book-keeping records of created and mounted 10057 volumes. 10058 .PP 10059 All mount and VFS options are submitted by the docker daemon via API, 10060 but you can also provide defaults on the command line as well as set 10061 path to the config file and cache directory or adjust logging verbosity. 10062 ## VFS - Virtual File System 10063 .PP 10064 This command uses the VFS layer. 10065 This adapts the cloud storage objects that rclone uses into something 10066 which looks much more like a disk filing system. 10067 .PP 10068 Cloud storage objects have lots of properties which aren\[aq]t like disk 10069 files - you can\[aq]t extend them or write to the middle of them, so the 10070 VFS layer has to deal with that. 10071 Because there is no one right way of doing this there are various 10072 options explained below. 10073 .PP 10074 The VFS layer also implements a directory cache - this caches info about 10075 files and directories (but not the data) in memory. 10076 .SS VFS Directory Cache 10077 .PP 10078 Using the \f[C]--dir-cache-time\f[R] flag, you can control how long a 10079 directory should be considered up to date and not refreshed from the 10080 backend. 10081 Changes made through the VFS will appear immediately or invalidate the 10082 cache. 10083 .IP 10084 .nf 10085 \f[C] 10086 --dir-cache-time duration Time to cache directory entries for (default 5m0s) 10087 --poll-interval duration Time to wait between polling for changes. Must be smaller than dir-cache-time. Only on supported remotes. Set to 0 to disable (default 1m0s) 10088 \f[R] 10089 .fi 10090 .PP 10091 However, changes made directly on the cloud storage by the web interface 10092 or a different copy of rclone will only be picked up once the directory 10093 cache expires if the backend configured does not support polling for 10094 changes. 10095 If the backend supports polling, changes will be picked up within the 10096 polling interval. 10097 .PP 10098 You can send a \f[C]SIGHUP\f[R] signal to rclone for it to flush all 10099 directory caches, regardless of how old they are. 10100 Assuming only one rclone instance is running, you can reset the cache 10101 like this: 10102 .IP 10103 .nf 10104 \f[C] 10105 kill -SIGHUP $(pidof rclone) 10106 \f[R] 10107 .fi 10108 .PP 10109 If you configure rclone with a remote control then you can use rclone rc 10110 to flush the whole directory cache: 10111 .IP 10112 .nf 10113 \f[C] 10114 rclone rc vfs/forget 10115 \f[R] 10116 .fi 10117 .PP 10118 Or individual files or directories: 10119 .IP 10120 .nf 10121 \f[C] 10122 rclone rc vfs/forget file=path/to/file dir=path/to/dir 10123 \f[R] 10124 .fi 10125 .SS VFS File Buffering 10126 .PP 10127 The \f[C]--buffer-size\f[R] flag determines the amount of memory, that 10128 will be used to buffer data in advance. 10129 .PP 10130 Each open file will try to keep the specified amount of data in memory 10131 at all times. 10132 The buffered data is bound to one open file and won\[aq]t be shared. 10133 .PP 10134 This flag is a upper limit for the used memory per open file. 10135 The buffer will only use memory for data that is downloaded but not not 10136 yet read. 10137 If the buffer is empty, only a small amount of memory will be used. 10138 .PP 10139 The maximum memory used by rclone for buffering can be up to 10140 \f[C]--buffer-size * open files\f[R]. 10141 .SS VFS File Caching 10142 .PP 10143 These flags control the VFS file caching options. 10144 File caching is necessary to make the VFS layer appear compatible with a 10145 normal file system. 10146 It can be disabled at the cost of some compatibility. 10147 .PP 10148 For example you\[aq]ll need to enable VFS caching if you want to read 10149 and write simultaneously to a file. 10150 See below for more details. 10151 .PP 10152 Note that the VFS cache is separate from the cache backend and you may 10153 find that you need one or the other or both. 10154 .IP 10155 .nf 10156 \f[C] 10157 --cache-dir string Directory rclone will use for caching. 10158 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off) 10159 --vfs-cache-max-age duration Max time since last access of objects in the cache (default 1h0m0s) 10160 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache (default off) 10161 --vfs-cache-min-free-space SizeSuffix Target minimum free space on the disk containing the cache (default off) 10162 --vfs-cache-poll-interval duration Interval to poll the cache for stale objects (default 1m0s) 10163 --vfs-write-back duration Time to writeback files after last use when using cache (default 5s) 10164 \f[R] 10165 .fi 10166 .PP 10167 If run with \f[C]-vv\f[R] rclone will print the location of the file 10168 cache. 10169 The files are stored in the user cache file area which is OS dependent 10170 but can be controlled with \f[C]--cache-dir\f[R] or setting the 10171 appropriate environment variable. 10172 .PP 10173 The cache has 4 different modes selected by \f[C]--vfs-cache-mode\f[R]. 10174 The higher the cache mode the more compatible rclone becomes at the cost 10175 of using disk space. 10176 .PP 10177 Note that files are written back to the remote only when they are closed 10178 and if they haven\[aq]t been accessed for \f[C]--vfs-write-back\f[R] 10179 seconds. 10180 If rclone is quit or dies with files that haven\[aq]t been uploaded, 10181 these will be uploaded next time rclone is run with the same flags. 10182 .PP 10183 If using \f[C]--vfs-cache-max-size\f[R] or 10184 \f[C]--vfs-cache-min-free-size\f[R] note that the cache may exceed these 10185 quotas for two reasons. 10186 Firstly because it is only checked every 10187 \f[C]--vfs-cache-poll-interval\f[R]. 10188 Secondly because open files cannot be evicted from the cache. 10189 When \f[C]--vfs-cache-max-size\f[R] or 10190 \f[C]--vfs-cache-min-free-size\f[R] is exceeded, rclone will attempt to 10191 evict the least accessed files from the cache first. 10192 rclone will start with files that haven\[aq]t been accessed for the 10193 longest. 10194 This cache flushing strategy is efficient and more relevant files are 10195 likely to remain cached. 10196 .PP 10197 The \f[C]--vfs-cache-max-age\f[R] will evict files from the cache after 10198 the set time since last access has passed. 10199 The default value of 1 hour will start evicting files from cache that 10200 haven\[aq]t been accessed for 1 hour. 10201 When a cached file is accessed the 1 hour timer is reset to 0 and will 10202 wait for 1 more hour before evicting. 10203 Specify the time with standard notation, s, m, h, d, w . 10204 .PP 10205 You \f[B]should not\f[R] run two copies of rclone using the same VFS 10206 cache with the same or overlapping remotes if using 10207 \f[C]--vfs-cache-mode > off\f[R]. 10208 This can potentially cause data corruption if you do. 10209 You can work around this by giving each rclone its own cache hierarchy 10210 with \f[C]--cache-dir\f[R]. 10211 You don\[aq]t need to worry about this if the remotes in use don\[aq]t 10212 overlap. 10213 .SS --vfs-cache-mode off 10214 .PP 10215 In this mode (the default) the cache will read directly from the remote 10216 and write directly to the remote without caching anything on disk. 10217 .PP 10218 This will mean some operations are not possible 10219 .IP \[bu] 2 10220 Files can\[aq]t be opened for both read AND write 10221 .IP \[bu] 2 10222 Files opened for write can\[aq]t be seeked 10223 .IP \[bu] 2 10224 Existing files opened for write must have O_TRUNC set 10225 .IP \[bu] 2 10226 Files open for read with O_TRUNC will be opened write only 10227 .IP \[bu] 2 10228 Files open for write only will behave as if O_TRUNC was supplied 10229 .IP \[bu] 2 10230 Open modes O_APPEND, O_TRUNC are ignored 10231 .IP \[bu] 2 10232 If an upload fails it can\[aq]t be retried 10233 .SS --vfs-cache-mode minimal 10234 .PP 10235 This is very similar to \[dq]off\[dq] except that files opened for read 10236 AND write will be buffered to disk. 10237 This means that files opened for write will be a lot more compatible, 10238 but uses the minimal disk space. 10239 .PP 10240 These operations are not possible 10241 .IP \[bu] 2 10242 Files opened for write only can\[aq]t be seeked 10243 .IP \[bu] 2 10244 Existing files opened for write must have O_TRUNC set 10245 .IP \[bu] 2 10246 Files opened for write only will ignore O_APPEND, O_TRUNC 10247 .IP \[bu] 2 10248 If an upload fails it can\[aq]t be retried 10249 .SS --vfs-cache-mode writes 10250 .PP 10251 In this mode files opened for read only are still read directly from the 10252 remote, write only and read/write files are buffered to disk first. 10253 .PP 10254 This mode should support all normal file system operations. 10255 .PP 10256 If an upload fails it will be retried at exponentially increasing 10257 intervals up to 1 minute. 10258 .SS --vfs-cache-mode full 10259 .PP 10260 In this mode all reads and writes are buffered to and from disk. 10261 When data is read from the remote this is buffered to disk as well. 10262 .PP 10263 In this mode the files in the cache will be sparse files and rclone will 10264 keep track of which bits of the files it has downloaded. 10265 .PP 10266 So if an application only reads the starts of each file, then rclone 10267 will only buffer the start of the file. 10268 These files will appear to be their full size in the cache, but they 10269 will be sparse files with only the data that has been downloaded present 10270 in them. 10271 .PP 10272 This mode should support all normal file system operations and is 10273 otherwise identical to \f[C]--vfs-cache-mode\f[R] writes. 10274 .PP 10275 When reading a file rclone will read \f[C]--buffer-size\f[R] plus 10276 \f[C]--vfs-read-ahead\f[R] bytes ahead. 10277 The \f[C]--buffer-size\f[R] is buffered in memory whereas the 10278 \f[C]--vfs-read-ahead\f[R] is buffered on disk. 10279 .PP 10280 When using this mode it is recommended that \f[C]--buffer-size\f[R] is 10281 not set too large and \f[C]--vfs-read-ahead\f[R] is set large if 10282 required. 10283 .PP 10284 \f[B]IMPORTANT\f[R] not all file systems support sparse files. 10285 In particular FAT/exFAT do not. 10286 Rclone will perform very badly if the cache directory is on a filesystem 10287 which doesn\[aq]t support sparse files and it will log an ERROR message 10288 if one is detected. 10289 .SS Fingerprinting 10290 .PP 10291 Various parts of the VFS use fingerprinting to see if a local file copy 10292 has changed relative to a remote file. 10293 Fingerprints are made from: 10294 .IP \[bu] 2 10295 size 10296 .IP \[bu] 2 10297 modification time 10298 .IP \[bu] 2 10299 hash 10300 .PP 10301 where available on an object. 10302 .PP 10303 On some backends some of these attributes are slow to read (they take an 10304 extra API call per object, or extra work per object). 10305 .PP 10306 For example \f[C]hash\f[R] is slow with the \f[C]local\f[R] and 10307 \f[C]sftp\f[R] backends as they have to read the entire file and hash 10308 it, and \f[C]modtime\f[R] is slow with the \f[C]s3\f[R], 10309 \f[C]swift\f[R], \f[C]ftp\f[R] and \f[C]qinqstor\f[R] backends because 10310 they need to do an extra API call to fetch it. 10311 .PP 10312 If you use the \f[C]--vfs-fast-fingerprint\f[R] flag then rclone will 10313 not include the slow operations in the fingerprint. 10314 This makes the fingerprinting less accurate but much faster and will 10315 improve the opening time of cached files. 10316 .PP 10317 If you are running a vfs cache over \f[C]local\f[R], \f[C]s3\f[R] or 10318 \f[C]swift\f[R] backends then using this flag is recommended. 10319 .PP 10320 Note that if you change the value of this flag, the fingerprints of the 10321 files in the cache may be invalidated and the files will need to be 10322 downloaded again. 10323 .SS VFS Chunked Reading 10324 .PP 10325 When rclone reads files from a remote it reads them in chunks. 10326 This means that rather than requesting the whole file rclone reads the 10327 chunk specified. 10328 This can reduce the used download quota for some remotes by requesting 10329 only chunks from the remote that are actually read, at the cost of an 10330 increased number of requests. 10331 .PP 10332 These flags control the chunking: 10333 .IP 10334 .nf 10335 \f[C] 10336 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks (default 128M) 10337 --vfs-read-chunk-size-limit SizeSuffix Max chunk doubling size (default off) 10338 \f[R] 10339 .fi 10340 .PP 10341 Rclone will start reading a chunk of size 10342 \f[C]--vfs-read-chunk-size\f[R], and then double the size for each read. 10343 When \f[C]--vfs-read-chunk-size-limit\f[R] is specified, and greater 10344 than \f[C]--vfs-read-chunk-size\f[R], the chunk size for each open file 10345 will get doubled only until the specified value is reached. 10346 If the value is \[dq]off\[dq], which is the default, the limit is 10347 disabled and the chunk size will grow indefinitely. 10348 .PP 10349 With \f[C]--vfs-read-chunk-size 100M\f[R] and 10350 \f[C]--vfs-read-chunk-size-limit 0\f[R] the following parts will be 10351 downloaded: 0-100M, 100M-200M, 200M-300M, 300M-400M and so on. 10352 When \f[C]--vfs-read-chunk-size-limit 500M\f[R] is specified, the result 10353 would be 0-100M, 100M-300M, 300M-700M, 700M-1200M, 1200M-1700M and so 10354 on. 10355 .PP 10356 Setting \f[C]--vfs-read-chunk-size\f[R] to \f[C]0\f[R] or \[dq]off\[dq] 10357 disables chunked reading. 10358 .SS VFS Performance 10359 .PP 10360 These flags may be used to enable/disable features of the VFS for 10361 performance or other reasons. 10362 See also the chunked reading feature. 10363 .PP 10364 In particular S3 and Swift benefit hugely from the 10365 \f[C]--no-modtime\f[R] flag (or use \f[C]--use-server-modtime\f[R] for a 10366 slightly different effect) as each read of the modification time takes a 10367 transaction. 10368 .IP 10369 .nf 10370 \f[C] 10371 --no-checksum Don\[aq]t compare checksums on up/download. 10372 --no-modtime Don\[aq]t read/write the modification time (can speed things up). 10373 --no-seek Don\[aq]t allow seeking in files. 10374 --read-only Only allow read-only access. 10375 \f[R] 10376 .fi 10377 .PP 10378 Sometimes rclone is delivered reads or writes out of order. 10379 Rather than seeking rclone will wait a short time for the in sequence 10380 read or write to come in. 10381 These flags only come into effect when not using an on disk cache file. 10382 .IP 10383 .nf 10384 \f[C] 10385 --vfs-read-wait duration Time to wait for in-sequence read before seeking (default 20ms) 10386 --vfs-write-wait duration Time to wait for in-sequence write before giving error (default 1s) 10387 \f[R] 10388 .fi 10389 .PP 10390 When using VFS write caching (\f[C]--vfs-cache-mode\f[R] with value 10391 writes or full), the global flag \f[C]--transfers\f[R] can be set to 10392 adjust the number of parallel uploads of modified files from the cache 10393 (the related global flag \f[C]--checkers\f[R] has no effect on the VFS). 10394 .IP 10395 .nf 10396 \f[C] 10397 --transfers int Number of file transfers to run in parallel (default 4) 10398 \f[R] 10399 .fi 10400 .SS VFS Case Sensitivity 10401 .PP 10402 Linux file systems are case-sensitive: two files can differ only by 10403 case, and the exact case must be used when opening a file. 10404 .PP 10405 File systems in modern Windows are case-insensitive but case-preserving: 10406 although existing files can be opened using any case, the exact case 10407 used to create the file is preserved and available for programs to 10408 query. 10409 It is not allowed for two files in the same directory to differ only by 10410 case. 10411 .PP 10412 Usually file systems on macOS are case-insensitive. 10413 It is possible to make macOS file systems case-sensitive but that is not 10414 the default. 10415 .PP 10416 The \f[C]--vfs-case-insensitive\f[R] VFS flag controls how rclone 10417 handles these two cases. 10418 If its value is \[dq]false\[dq], rclone passes file names to the remote 10419 as-is. 10420 If the flag is \[dq]true\[dq] (or appears without a value on the command 10421 line), rclone may perform a \[dq]fixup\[dq] as explained below. 10422 .PP 10423 The user may specify a file name to open/delete/rename/etc with a case 10424 different than what is stored on the remote. 10425 If an argument refers to an existing file with exactly the same name, 10426 then the case of the existing file on the disk will be used. 10427 However, if a file name with exactly the same name is not found but a 10428 name differing only by case exists, rclone will transparently fixup the 10429 name. 10430 This fixup happens only when an existing file is requested. 10431 Case sensitivity of file names created anew by rclone is controlled by 10432 the underlying remote. 10433 .PP 10434 Note that case sensitivity of the operating system running rclone (the 10435 target) may differ from case sensitivity of a file system presented by 10436 rclone (the source). 10437 The flag controls whether \[dq]fixup\[dq] is performed to satisfy the 10438 target. 10439 .PP 10440 If the flag is not provided on the command line, then its default value 10441 depends on the operating system where rclone runs: \[dq]true\[dq] on 10442 Windows and macOS, \[dq]false\[dq] otherwise. 10443 If the flag is provided without a value, then it is \[dq]true\[dq]. 10444 .PP 10445 The \f[C]--no-unicode-normalization\f[R] flag controls whether a similar 10446 \[dq]fixup\[dq] is performed for filenames that differ but are 10447 canonically 10448 equivalent (https://en.wikipedia.org/wiki/Unicode_equivalence) with 10449 respect to unicode. 10450 Unicode normalization can be particularly helpful for users of macOS, 10451 which prefers form NFD instead of the NFC used by most other platforms. 10452 It is therefore highly recommended to keep the default of 10453 \f[C]false\f[R] on macOS, to avoid encoding compatibility issues. 10454 .PP 10455 In the (probably unlikely) event that a directory has multiple duplicate 10456 filenames after applying case and unicode normalization, the 10457 \f[C]--vfs-block-norm-dupes\f[R] flag allows hiding these duplicates. 10458 This comes with a performance tradeoff, as rclone will have to scan the 10459 entire directory for duplicates when listing a directory. 10460 For this reason, it is recommended to leave this disabled if not needed. 10461 However, macOS users may wish to consider using it, as otherwise, if a 10462 remote directory contains both NFC and NFD versions of the same 10463 filename, an odd situation will occur: both versions of the file will be 10464 visible in the mount, and both will appear to be editable, however, 10465 editing either version will actually result in only the NFD version 10466 getting edited under the hood. 10467 \f[C]--vfs-block- norm-dupes\f[R] prevents this confusion by detecting 10468 this scenario, hiding the duplicates, and logging an error, similar to 10469 how this is handled in \f[C]rclone sync\f[R]. 10470 .SS VFS Disk Options 10471 .PP 10472 This flag allows you to manually set the statistics about the filing 10473 system. 10474 It can be useful when those statistics cannot be read correctly 10475 automatically. 10476 .IP 10477 .nf 10478 \f[C] 10479 --vfs-disk-space-total-size Manually set the total disk space size (example: 256G, default: -1) 10480 \f[R] 10481 .fi 10482 .SS Alternate report of used bytes 10483 .PP 10484 Some backends, most notably S3, do not report the amount of bytes used. 10485 If you need this information to be available when running \f[C]df\f[R] 10486 on the filesystem, then pass the flag \f[C]--vfs-used-is-size\f[R] to 10487 rclone. 10488 With this flag set, instead of relying on the backend to report this 10489 information, rclone will scan the whole remote similar to 10490 \f[C]rclone size\f[R] and compute the total used space itself. 10491 .PP 10492 \f[I]WARNING.\f[R] Contrary to \f[C]rclone size\f[R], this flag ignores 10493 filters so that the result is accurate. 10494 However, this is very inefficient and may cost lots of API calls 10495 resulting in extra charges. 10496 Use it as a last resort and only with caching. 10497 .IP 10498 .nf 10499 \f[C] 10500 rclone serve docker [flags] 10501 \f[R] 10502 .fi 10503 .SS Options 10504 .IP 10505 .nf 10506 \f[C] 10507 --allow-non-empty Allow mounting over a non-empty directory (not supported on Windows) 10508 --allow-other Allow access to other users (not supported on Windows) 10509 --allow-root Allow access to root user (not supported on Windows) 10510 --async-read Use asynchronous reads (not supported on Windows) (default true) 10511 --attr-timeout Duration Time for which file/directory attributes are cached (default 1s) 10512 --base-dir string Base directory for volumes (default \[dq]/var/lib/docker-volumes/rclone\[dq]) 10513 --daemon Run mount in background and exit parent process (as background output is suppressed, use --log-file with --log-format=pid,... to monitor) (not supported on Windows) 10514 --daemon-timeout Duration Time limit for rclone to respond to kernel (not supported on Windows) (default 0s) 10515 --daemon-wait Duration Time to wait for ready mount from daemon (maximum time on Linux, constant sleep time on OSX/BSD) (not supported on Windows) (default 1m0s) 10516 --debug-fuse Debug the FUSE internals - needs -v 10517 --default-permissions Makes kernel enforce access control based on the file mode (not supported on Windows) 10518 --devname string Set the device name - default is remote:path 10519 --dir-cache-time Duration Time to cache directory entries for (default 5m0s) 10520 --dir-perms FileMode Directory permissions (default 0777) 10521 --file-perms FileMode File permissions (default 0666) 10522 --forget-state Skip restoring previous state 10523 --fuse-flag stringArray Flags or arguments to be passed direct to libfuse/WinFsp (repeat if required) 10524 --gid uint32 Override the gid field set by the filesystem (not supported on Windows) (default 1000) 10525 -h, --help help for docker 10526 --max-read-ahead SizeSuffix The number of bytes that can be prefetched for sequential reads (not supported on Windows) (default 128Ki) 10527 --mount-case-insensitive Tristate Tell the OS the mount is case insensitive (true) or sensitive (false) regardless of the backend (auto) (default unset) 10528 --network-mode Mount as remote network drive, instead of fixed disk drive (supported on Windows only) 10529 --no-checksum Don\[aq]t compare checksums on up/download 10530 --no-modtime Don\[aq]t read/write the modification time (can speed things up) 10531 --no-seek Don\[aq]t allow seeking in files 10532 --no-spec Do not write spec file 10533 --noappledouble Ignore Apple Double (._) and .DS_Store files (supported on OSX only) (default true) 10534 --noapplexattr Ignore all \[dq]com.apple.*\[dq] extended attributes (supported on OSX only) 10535 -o, --option stringArray Option for libfuse/WinFsp (repeat if required) 10536 --poll-interval Duration Time to wait between polling for changes, must be smaller than dir-cache-time and only on supported remotes (set 0 to disable) (default 1m0s) 10537 --read-only Only allow read-only access 10538 --socket-addr string Address <host:port> or absolute path (default: /run/docker/plugins/rclone.sock) 10539 --socket-gid int GID for unix socket (default: current process GID) (default 1000) 10540 --uid uint32 Override the uid field set by the filesystem (not supported on Windows) (default 1000) 10541 --umask int Override the permission bits set by the filesystem (not supported on Windows) (default 2) 10542 --vfs-block-norm-dupes If duplicate filenames exist in the same directory (after normalization), log an error and hide the duplicates (may have a performance cost) 10543 --vfs-cache-max-age Duration Max time since last access of objects in the cache (default 1h0m0s) 10544 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache (default off) 10545 --vfs-cache-min-free-space SizeSuffix Target minimum free space on the disk containing the cache (default off) 10546 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off) 10547 --vfs-cache-poll-interval Duration Interval to poll the cache for stale objects (default 1m0s) 10548 --vfs-case-insensitive If a file name not found, find a case insensitive match 10549 --vfs-disk-space-total-size SizeSuffix Specify the total space of disk (default off) 10550 --vfs-fast-fingerprint Use fast (less accurate) fingerprints for change detection 10551 --vfs-read-ahead SizeSuffix Extra read ahead over --buffer-size when using cache-mode full 10552 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks (default 128Mi) 10553 --vfs-read-chunk-size-limit SizeSuffix If greater than --vfs-read-chunk-size, double the chunk size after each chunk read, until the limit is reached (\[aq]off\[aq] is unlimited) (default off) 10554 --vfs-read-wait Duration Time to wait for in-sequence read before seeking (default 20ms) 10555 --vfs-refresh Refreshes the directory cache recursively in the background on start 10556 --vfs-used-is-size rclone size Use the rclone size algorithm for Used size 10557 --vfs-write-back Duration Time to writeback files after last use when using cache (default 5s) 10558 --vfs-write-wait Duration Time to wait for in-sequence write before giving error (default 1s) 10559 --volname string Set the volume name (supported on Windows and OSX only) 10560 --write-back-cache Makes kernel buffer writes before sending them to rclone (without this, writethrough caching is used) (not supported on Windows) 10561 \f[R] 10562 .fi 10563 .SS Filter Options 10564 .PP 10565 Flags for filtering directory listings. 10566 .IP 10567 .nf 10568 \f[C] 10569 --delete-excluded Delete files on dest excluded from sync 10570 --exclude stringArray Exclude files matching pattern 10571 --exclude-from stringArray Read file exclude patterns from file (use - to read from stdin) 10572 --exclude-if-present stringArray Exclude directories if filename is present 10573 --files-from stringArray Read list of source-file names from file (use - to read from stdin) 10574 --files-from-raw stringArray Read list of source-file names from file without any processing of lines (use - to read from stdin) 10575 -f, --filter stringArray Add a file filtering rule 10576 --filter-from stringArray Read file filtering patterns from a file (use - to read from stdin) 10577 --ignore-case Ignore case in filters (case insensitive) 10578 --include stringArray Include files matching pattern 10579 --include-from stringArray Read file include patterns from file (use - to read from stdin) 10580 --max-age Duration Only transfer files younger than this in s or suffix ms|s|m|h|d|w|M|y (default off) 10581 --max-depth int If set limits the recursion depth to this (default -1) 10582 --max-size SizeSuffix Only transfer files smaller than this in KiB or suffix B|K|M|G|T|P (default off) 10583 --metadata-exclude stringArray Exclude metadatas matching pattern 10584 --metadata-exclude-from stringArray Read metadata exclude patterns from file (use - to read from stdin) 10585 --metadata-filter stringArray Add a metadata filtering rule 10586 --metadata-filter-from stringArray Read metadata filtering patterns from a file (use - to read from stdin) 10587 --metadata-include stringArray Include metadatas matching pattern 10588 --metadata-include-from stringArray Read metadata include patterns from file (use - to read from stdin) 10589 --min-age Duration Only transfer files older than this in s or suffix ms|s|m|h|d|w|M|y (default off) 10590 --min-size SizeSuffix Only transfer files bigger than this in KiB or suffix B|K|M|G|T|P (default off) 10591 \f[R] 10592 .fi 10593 .PP 10594 See the global flags page (https://rclone.org/flags/) for global options 10595 not listed here. 10596 .SH SEE ALSO 10597 .IP \[bu] 2 10598 rclone serve (https://rclone.org/commands/rclone_serve/) - Serve a 10599 remote over a protocol. 10600 .SH rclone serve ftp 10601 .PP 10602 Serve remote:path over FTP. 10603 .SS Synopsis 10604 .PP 10605 Run a basic FTP server to serve a remote over FTP protocol. 10606 This can be viewed with a FTP client or you can make a remote of type 10607 FTP to read and write it. 10608 .SS Server options 10609 .PP 10610 Use --addr to specify which IP address and port the server should listen 10611 on, e.g. 10612 --addr 1.2.3.4:8000 or --addr :8080 to listen to all IPs. 10613 By default it only listens on localhost. 10614 You can use port :0 to let the OS choose an available port. 10615 .PP 10616 If you set --addr to listen on a public or LAN accessible IP address 10617 then using Authentication is advised - see the next section for info. 10618 .SS Authentication 10619 .PP 10620 By default this will serve files without needing a login. 10621 .PP 10622 You can set a single username and password with the --user and --pass 10623 flags. 10624 ## VFS - Virtual File System 10625 .PP 10626 This command uses the VFS layer. 10627 This adapts the cloud storage objects that rclone uses into something 10628 which looks much more like a disk filing system. 10629 .PP 10630 Cloud storage objects have lots of properties which aren\[aq]t like disk 10631 files - you can\[aq]t extend them or write to the middle of them, so the 10632 VFS layer has to deal with that. 10633 Because there is no one right way of doing this there are various 10634 options explained below. 10635 .PP 10636 The VFS layer also implements a directory cache - this caches info about 10637 files and directories (but not the data) in memory. 10638 .SS VFS Directory Cache 10639 .PP 10640 Using the \f[C]--dir-cache-time\f[R] flag, you can control how long a 10641 directory should be considered up to date and not refreshed from the 10642 backend. 10643 Changes made through the VFS will appear immediately or invalidate the 10644 cache. 10645 .IP 10646 .nf 10647 \f[C] 10648 --dir-cache-time duration Time to cache directory entries for (default 5m0s) 10649 --poll-interval duration Time to wait between polling for changes. Must be smaller than dir-cache-time. Only on supported remotes. Set to 0 to disable (default 1m0s) 10650 \f[R] 10651 .fi 10652 .PP 10653 However, changes made directly on the cloud storage by the web interface 10654 or a different copy of rclone will only be picked up once the directory 10655 cache expires if the backend configured does not support polling for 10656 changes. 10657 If the backend supports polling, changes will be picked up within the 10658 polling interval. 10659 .PP 10660 You can send a \f[C]SIGHUP\f[R] signal to rclone for it to flush all 10661 directory caches, regardless of how old they are. 10662 Assuming only one rclone instance is running, you can reset the cache 10663 like this: 10664 .IP 10665 .nf 10666 \f[C] 10667 kill -SIGHUP $(pidof rclone) 10668 \f[R] 10669 .fi 10670 .PP 10671 If you configure rclone with a remote control then you can use rclone rc 10672 to flush the whole directory cache: 10673 .IP 10674 .nf 10675 \f[C] 10676 rclone rc vfs/forget 10677 \f[R] 10678 .fi 10679 .PP 10680 Or individual files or directories: 10681 .IP 10682 .nf 10683 \f[C] 10684 rclone rc vfs/forget file=path/to/file dir=path/to/dir 10685 \f[R] 10686 .fi 10687 .SS VFS File Buffering 10688 .PP 10689 The \f[C]--buffer-size\f[R] flag determines the amount of memory, that 10690 will be used to buffer data in advance. 10691 .PP 10692 Each open file will try to keep the specified amount of data in memory 10693 at all times. 10694 The buffered data is bound to one open file and won\[aq]t be shared. 10695 .PP 10696 This flag is a upper limit for the used memory per open file. 10697 The buffer will only use memory for data that is downloaded but not not 10698 yet read. 10699 If the buffer is empty, only a small amount of memory will be used. 10700 .PP 10701 The maximum memory used by rclone for buffering can be up to 10702 \f[C]--buffer-size * open files\f[R]. 10703 .SS VFS File Caching 10704 .PP 10705 These flags control the VFS file caching options. 10706 File caching is necessary to make the VFS layer appear compatible with a 10707 normal file system. 10708 It can be disabled at the cost of some compatibility. 10709 .PP 10710 For example you\[aq]ll need to enable VFS caching if you want to read 10711 and write simultaneously to a file. 10712 See below for more details. 10713 .PP 10714 Note that the VFS cache is separate from the cache backend and you may 10715 find that you need one or the other or both. 10716 .IP 10717 .nf 10718 \f[C] 10719 --cache-dir string Directory rclone will use for caching. 10720 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off) 10721 --vfs-cache-max-age duration Max time since last access of objects in the cache (default 1h0m0s) 10722 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache (default off) 10723 --vfs-cache-min-free-space SizeSuffix Target minimum free space on the disk containing the cache (default off) 10724 --vfs-cache-poll-interval duration Interval to poll the cache for stale objects (default 1m0s) 10725 --vfs-write-back duration Time to writeback files after last use when using cache (default 5s) 10726 \f[R] 10727 .fi 10728 .PP 10729 If run with \f[C]-vv\f[R] rclone will print the location of the file 10730 cache. 10731 The files are stored in the user cache file area which is OS dependent 10732 but can be controlled with \f[C]--cache-dir\f[R] or setting the 10733 appropriate environment variable. 10734 .PP 10735 The cache has 4 different modes selected by \f[C]--vfs-cache-mode\f[R]. 10736 The higher the cache mode the more compatible rclone becomes at the cost 10737 of using disk space. 10738 .PP 10739 Note that files are written back to the remote only when they are closed 10740 and if they haven\[aq]t been accessed for \f[C]--vfs-write-back\f[R] 10741 seconds. 10742 If rclone is quit or dies with files that haven\[aq]t been uploaded, 10743 these will be uploaded next time rclone is run with the same flags. 10744 .PP 10745 If using \f[C]--vfs-cache-max-size\f[R] or 10746 \f[C]--vfs-cache-min-free-size\f[R] note that the cache may exceed these 10747 quotas for two reasons. 10748 Firstly because it is only checked every 10749 \f[C]--vfs-cache-poll-interval\f[R]. 10750 Secondly because open files cannot be evicted from the cache. 10751 When \f[C]--vfs-cache-max-size\f[R] or 10752 \f[C]--vfs-cache-min-free-size\f[R] is exceeded, rclone will attempt to 10753 evict the least accessed files from the cache first. 10754 rclone will start with files that haven\[aq]t been accessed for the 10755 longest. 10756 This cache flushing strategy is efficient and more relevant files are 10757 likely to remain cached. 10758 .PP 10759 The \f[C]--vfs-cache-max-age\f[R] will evict files from the cache after 10760 the set time since last access has passed. 10761 The default value of 1 hour will start evicting files from cache that 10762 haven\[aq]t been accessed for 1 hour. 10763 When a cached file is accessed the 1 hour timer is reset to 0 and will 10764 wait for 1 more hour before evicting. 10765 Specify the time with standard notation, s, m, h, d, w . 10766 .PP 10767 You \f[B]should not\f[R] run two copies of rclone using the same VFS 10768 cache with the same or overlapping remotes if using 10769 \f[C]--vfs-cache-mode > off\f[R]. 10770 This can potentially cause data corruption if you do. 10771 You can work around this by giving each rclone its own cache hierarchy 10772 with \f[C]--cache-dir\f[R]. 10773 You don\[aq]t need to worry about this if the remotes in use don\[aq]t 10774 overlap. 10775 .SS --vfs-cache-mode off 10776 .PP 10777 In this mode (the default) the cache will read directly from the remote 10778 and write directly to the remote without caching anything on disk. 10779 .PP 10780 This will mean some operations are not possible 10781 .IP \[bu] 2 10782 Files can\[aq]t be opened for both read AND write 10783 .IP \[bu] 2 10784 Files opened for write can\[aq]t be seeked 10785 .IP \[bu] 2 10786 Existing files opened for write must have O_TRUNC set 10787 .IP \[bu] 2 10788 Files open for read with O_TRUNC will be opened write only 10789 .IP \[bu] 2 10790 Files open for write only will behave as if O_TRUNC was supplied 10791 .IP \[bu] 2 10792 Open modes O_APPEND, O_TRUNC are ignored 10793 .IP \[bu] 2 10794 If an upload fails it can\[aq]t be retried 10795 .SS --vfs-cache-mode minimal 10796 .PP 10797 This is very similar to \[dq]off\[dq] except that files opened for read 10798 AND write will be buffered to disk. 10799 This means that files opened for write will be a lot more compatible, 10800 but uses the minimal disk space. 10801 .PP 10802 These operations are not possible 10803 .IP \[bu] 2 10804 Files opened for write only can\[aq]t be seeked 10805 .IP \[bu] 2 10806 Existing files opened for write must have O_TRUNC set 10807 .IP \[bu] 2 10808 Files opened for write only will ignore O_APPEND, O_TRUNC 10809 .IP \[bu] 2 10810 If an upload fails it can\[aq]t be retried 10811 .SS --vfs-cache-mode writes 10812 .PP 10813 In this mode files opened for read only are still read directly from the 10814 remote, write only and read/write files are buffered to disk first. 10815 .PP 10816 This mode should support all normal file system operations. 10817 .PP 10818 If an upload fails it will be retried at exponentially increasing 10819 intervals up to 1 minute. 10820 .SS --vfs-cache-mode full 10821 .PP 10822 In this mode all reads and writes are buffered to and from disk. 10823 When data is read from the remote this is buffered to disk as well. 10824 .PP 10825 In this mode the files in the cache will be sparse files and rclone will 10826 keep track of which bits of the files it has downloaded. 10827 .PP 10828 So if an application only reads the starts of each file, then rclone 10829 will only buffer the start of the file. 10830 These files will appear to be their full size in the cache, but they 10831 will be sparse files with only the data that has been downloaded present 10832 in them. 10833 .PP 10834 This mode should support all normal file system operations and is 10835 otherwise identical to \f[C]--vfs-cache-mode\f[R] writes. 10836 .PP 10837 When reading a file rclone will read \f[C]--buffer-size\f[R] plus 10838 \f[C]--vfs-read-ahead\f[R] bytes ahead. 10839 The \f[C]--buffer-size\f[R] is buffered in memory whereas the 10840 \f[C]--vfs-read-ahead\f[R] is buffered on disk. 10841 .PP 10842 When using this mode it is recommended that \f[C]--buffer-size\f[R] is 10843 not set too large and \f[C]--vfs-read-ahead\f[R] is set large if 10844 required. 10845 .PP 10846 \f[B]IMPORTANT\f[R] not all file systems support sparse files. 10847 In particular FAT/exFAT do not. 10848 Rclone will perform very badly if the cache directory is on a filesystem 10849 which doesn\[aq]t support sparse files and it will log an ERROR message 10850 if one is detected. 10851 .SS Fingerprinting 10852 .PP 10853 Various parts of the VFS use fingerprinting to see if a local file copy 10854 has changed relative to a remote file. 10855 Fingerprints are made from: 10856 .IP \[bu] 2 10857 size 10858 .IP \[bu] 2 10859 modification time 10860 .IP \[bu] 2 10861 hash 10862 .PP 10863 where available on an object. 10864 .PP 10865 On some backends some of these attributes are slow to read (they take an 10866 extra API call per object, or extra work per object). 10867 .PP 10868 For example \f[C]hash\f[R] is slow with the \f[C]local\f[R] and 10869 \f[C]sftp\f[R] backends as they have to read the entire file and hash 10870 it, and \f[C]modtime\f[R] is slow with the \f[C]s3\f[R], 10871 \f[C]swift\f[R], \f[C]ftp\f[R] and \f[C]qinqstor\f[R] backends because 10872 they need to do an extra API call to fetch it. 10873 .PP 10874 If you use the \f[C]--vfs-fast-fingerprint\f[R] flag then rclone will 10875 not include the slow operations in the fingerprint. 10876 This makes the fingerprinting less accurate but much faster and will 10877 improve the opening time of cached files. 10878 .PP 10879 If you are running a vfs cache over \f[C]local\f[R], \f[C]s3\f[R] or 10880 \f[C]swift\f[R] backends then using this flag is recommended. 10881 .PP 10882 Note that if you change the value of this flag, the fingerprints of the 10883 files in the cache may be invalidated and the files will need to be 10884 downloaded again. 10885 .SS VFS Chunked Reading 10886 .PP 10887 When rclone reads files from a remote it reads them in chunks. 10888 This means that rather than requesting the whole file rclone reads the 10889 chunk specified. 10890 This can reduce the used download quota for some remotes by requesting 10891 only chunks from the remote that are actually read, at the cost of an 10892 increased number of requests. 10893 .PP 10894 These flags control the chunking: 10895 .IP 10896 .nf 10897 \f[C] 10898 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks (default 128M) 10899 --vfs-read-chunk-size-limit SizeSuffix Max chunk doubling size (default off) 10900 \f[R] 10901 .fi 10902 .PP 10903 Rclone will start reading a chunk of size 10904 \f[C]--vfs-read-chunk-size\f[R], and then double the size for each read. 10905 When \f[C]--vfs-read-chunk-size-limit\f[R] is specified, and greater 10906 than \f[C]--vfs-read-chunk-size\f[R], the chunk size for each open file 10907 will get doubled only until the specified value is reached. 10908 If the value is \[dq]off\[dq], which is the default, the limit is 10909 disabled and the chunk size will grow indefinitely. 10910 .PP 10911 With \f[C]--vfs-read-chunk-size 100M\f[R] and 10912 \f[C]--vfs-read-chunk-size-limit 0\f[R] the following parts will be 10913 downloaded: 0-100M, 100M-200M, 200M-300M, 300M-400M and so on. 10914 When \f[C]--vfs-read-chunk-size-limit 500M\f[R] is specified, the result 10915 would be 0-100M, 100M-300M, 300M-700M, 700M-1200M, 1200M-1700M and so 10916 on. 10917 .PP 10918 Setting \f[C]--vfs-read-chunk-size\f[R] to \f[C]0\f[R] or \[dq]off\[dq] 10919 disables chunked reading. 10920 .SS VFS Performance 10921 .PP 10922 These flags may be used to enable/disable features of the VFS for 10923 performance or other reasons. 10924 See also the chunked reading feature. 10925 .PP 10926 In particular S3 and Swift benefit hugely from the 10927 \f[C]--no-modtime\f[R] flag (or use \f[C]--use-server-modtime\f[R] for a 10928 slightly different effect) as each read of the modification time takes a 10929 transaction. 10930 .IP 10931 .nf 10932 \f[C] 10933 --no-checksum Don\[aq]t compare checksums on up/download. 10934 --no-modtime Don\[aq]t read/write the modification time (can speed things up). 10935 --no-seek Don\[aq]t allow seeking in files. 10936 --read-only Only allow read-only access. 10937 \f[R] 10938 .fi 10939 .PP 10940 Sometimes rclone is delivered reads or writes out of order. 10941 Rather than seeking rclone will wait a short time for the in sequence 10942 read or write to come in. 10943 These flags only come into effect when not using an on disk cache file. 10944 .IP 10945 .nf 10946 \f[C] 10947 --vfs-read-wait duration Time to wait for in-sequence read before seeking (default 20ms) 10948 --vfs-write-wait duration Time to wait for in-sequence write before giving error (default 1s) 10949 \f[R] 10950 .fi 10951 .PP 10952 When using VFS write caching (\f[C]--vfs-cache-mode\f[R] with value 10953 writes or full), the global flag \f[C]--transfers\f[R] can be set to 10954 adjust the number of parallel uploads of modified files from the cache 10955 (the related global flag \f[C]--checkers\f[R] has no effect on the VFS). 10956 .IP 10957 .nf 10958 \f[C] 10959 --transfers int Number of file transfers to run in parallel (default 4) 10960 \f[R] 10961 .fi 10962 .SS VFS Case Sensitivity 10963 .PP 10964 Linux file systems are case-sensitive: two files can differ only by 10965 case, and the exact case must be used when opening a file. 10966 .PP 10967 File systems in modern Windows are case-insensitive but case-preserving: 10968 although existing files can be opened using any case, the exact case 10969 used to create the file is preserved and available for programs to 10970 query. 10971 It is not allowed for two files in the same directory to differ only by 10972 case. 10973 .PP 10974 Usually file systems on macOS are case-insensitive. 10975 It is possible to make macOS file systems case-sensitive but that is not 10976 the default. 10977 .PP 10978 The \f[C]--vfs-case-insensitive\f[R] VFS flag controls how rclone 10979 handles these two cases. 10980 If its value is \[dq]false\[dq], rclone passes file names to the remote 10981 as-is. 10982 If the flag is \[dq]true\[dq] (or appears without a value on the command 10983 line), rclone may perform a \[dq]fixup\[dq] as explained below. 10984 .PP 10985 The user may specify a file name to open/delete/rename/etc with a case 10986 different than what is stored on the remote. 10987 If an argument refers to an existing file with exactly the same name, 10988 then the case of the existing file on the disk will be used. 10989 However, if a file name with exactly the same name is not found but a 10990 name differing only by case exists, rclone will transparently fixup the 10991 name. 10992 This fixup happens only when an existing file is requested. 10993 Case sensitivity of file names created anew by rclone is controlled by 10994 the underlying remote. 10995 .PP 10996 Note that case sensitivity of the operating system running rclone (the 10997 target) may differ from case sensitivity of a file system presented by 10998 rclone (the source). 10999 The flag controls whether \[dq]fixup\[dq] is performed to satisfy the 11000 target. 11001 .PP 11002 If the flag is not provided on the command line, then its default value 11003 depends on the operating system where rclone runs: \[dq]true\[dq] on 11004 Windows and macOS, \[dq]false\[dq] otherwise. 11005 If the flag is provided without a value, then it is \[dq]true\[dq]. 11006 .PP 11007 The \f[C]--no-unicode-normalization\f[R] flag controls whether a similar 11008 \[dq]fixup\[dq] is performed for filenames that differ but are 11009 canonically 11010 equivalent (https://en.wikipedia.org/wiki/Unicode_equivalence) with 11011 respect to unicode. 11012 Unicode normalization can be particularly helpful for users of macOS, 11013 which prefers form NFD instead of the NFC used by most other platforms. 11014 It is therefore highly recommended to keep the default of 11015 \f[C]false\f[R] on macOS, to avoid encoding compatibility issues. 11016 .PP 11017 In the (probably unlikely) event that a directory has multiple duplicate 11018 filenames after applying case and unicode normalization, the 11019 \f[C]--vfs-block-norm-dupes\f[R] flag allows hiding these duplicates. 11020 This comes with a performance tradeoff, as rclone will have to scan the 11021 entire directory for duplicates when listing a directory. 11022 For this reason, it is recommended to leave this disabled if not needed. 11023 However, macOS users may wish to consider using it, as otherwise, if a 11024 remote directory contains both NFC and NFD versions of the same 11025 filename, an odd situation will occur: both versions of the file will be 11026 visible in the mount, and both will appear to be editable, however, 11027 editing either version will actually result in only the NFD version 11028 getting edited under the hood. 11029 \f[C]--vfs-block- norm-dupes\f[R] prevents this confusion by detecting 11030 this scenario, hiding the duplicates, and logging an error, similar to 11031 how this is handled in \f[C]rclone sync\f[R]. 11032 .SS VFS Disk Options 11033 .PP 11034 This flag allows you to manually set the statistics about the filing 11035 system. 11036 It can be useful when those statistics cannot be read correctly 11037 automatically. 11038 .IP 11039 .nf 11040 \f[C] 11041 --vfs-disk-space-total-size Manually set the total disk space size (example: 256G, default: -1) 11042 \f[R] 11043 .fi 11044 .SS Alternate report of used bytes 11045 .PP 11046 Some backends, most notably S3, do not report the amount of bytes used. 11047 If you need this information to be available when running \f[C]df\f[R] 11048 on the filesystem, then pass the flag \f[C]--vfs-used-is-size\f[R] to 11049 rclone. 11050 With this flag set, instead of relying on the backend to report this 11051 information, rclone will scan the whole remote similar to 11052 \f[C]rclone size\f[R] and compute the total used space itself. 11053 .PP 11054 \f[I]WARNING.\f[R] Contrary to \f[C]rclone size\f[R], this flag ignores 11055 filters so that the result is accurate. 11056 However, this is very inefficient and may cost lots of API calls 11057 resulting in extra charges. 11058 Use it as a last resort and only with caching. 11059 .SS Auth Proxy 11060 .PP 11061 If you supply the parameter \f[C]--auth-proxy /path/to/program\f[R] then 11062 rclone will use that program to generate backends on the fly which then 11063 are used to authenticate incoming requests. 11064 This uses a simple JSON based protocol with input on STDIN and output on 11065 STDOUT. 11066 .PP 11067 \f[B]PLEASE NOTE:\f[R] \f[C]--auth-proxy\f[R] and 11068 \f[C]--authorized-keys\f[R] cannot be used together, if 11069 \f[C]--auth-proxy\f[R] is set the authorized keys option will be 11070 ignored. 11071 .PP 11072 There is an example program 11073 bin/test_proxy.py (https://github.com/artpar/artpar/blob/master/bin/test_proxy.py) 11074 in the rclone source code. 11075 .PP 11076 The program\[aq]s job is to take a \f[C]user\f[R] and \f[C]pass\f[R] on 11077 the input and turn those into the config for a backend on STDOUT in JSON 11078 format. 11079 This config will have any default parameters for the backend added, but 11080 it won\[aq]t use configuration from environment variables or command 11081 line options - it is the job of the proxy program to make a complete 11082 config. 11083 .PP 11084 This config generated must have this extra parameter - \f[C]_root\f[R] - 11085 root to use for the backend 11086 .PP 11087 And it may have this parameter - \f[C]_obscure\f[R] - comma separated 11088 strings for parameters to obscure 11089 .PP 11090 If password authentication was used by the client, input to the proxy 11091 process (on STDIN) would look similar to this: 11092 .IP 11093 .nf 11094 \f[C] 11095 { 11096 \[dq]user\[dq]: \[dq]me\[dq], 11097 \[dq]pass\[dq]: \[dq]mypassword\[dq] 11098 } 11099 \f[R] 11100 .fi 11101 .PP 11102 If public-key authentication was used by the client, input to the proxy 11103 process (on STDIN) would look similar to this: 11104 .IP 11105 .nf 11106 \f[C] 11107 { 11108 \[dq]user\[dq]: \[dq]me\[dq], 11109 \[dq]public_key\[dq]: \[dq]AAAAB3NzaC1yc2EAAAADAQABAAABAQDuwESFdAe14hVS6omeyX7edc...JQdf\[dq] 11110 } 11111 \f[R] 11112 .fi 11113 .PP 11114 And as an example return this on STDOUT 11115 .IP 11116 .nf 11117 \f[C] 11118 { 11119 \[dq]type\[dq]: \[dq]sftp\[dq], 11120 \[dq]_root\[dq]: \[dq]\[dq], 11121 \[dq]_obscure\[dq]: \[dq]pass\[dq], 11122 \[dq]user\[dq]: \[dq]me\[dq], 11123 \[dq]pass\[dq]: \[dq]mypassword\[dq], 11124 \[dq]host\[dq]: \[dq]sftp.example.com\[dq] 11125 } 11126 \f[R] 11127 .fi 11128 .PP 11129 This would mean that an SFTP backend would be created on the fly for the 11130 \f[C]user\f[R] and \f[C]pass\f[R]/\f[C]public_key\f[R] returned in the 11131 output to the host given. 11132 Note that since \f[C]_obscure\f[R] is set to \f[C]pass\f[R], rclone will 11133 obscure the \f[C]pass\f[R] parameter before creating the backend (which 11134 is required for sftp backends). 11135 .PP 11136 The program can manipulate the supplied \f[C]user\f[R] in any way, for 11137 example to make proxy to many different sftp backends, you could make 11138 the \f[C]user\f[R] be \f[C]user\[at]example.com\f[R] and then set the 11139 \f[C]host\f[R] to \f[C]example.com\f[R] in the output and the user to 11140 \f[C]user\f[R]. 11141 For security you\[aq]d probably want to restrict the \f[C]host\f[R] to a 11142 limited list. 11143 .PP 11144 Note that an internal cache is keyed on \f[C]user\f[R] so only use that 11145 for configuration, don\[aq]t use \f[C]pass\f[R] or \f[C]public_key\f[R]. 11146 This also means that if a user\[aq]s password or public-key is changed 11147 the cache will need to expire (which takes 5 mins) before it takes 11148 effect. 11149 .PP 11150 This can be used to build general purpose proxies to any kind of backend 11151 that rclone supports. 11152 .IP 11153 .nf 11154 \f[C] 11155 rclone serve ftp remote:path [flags] 11156 \f[R] 11157 .fi 11158 .SS Options 11159 .IP 11160 .nf 11161 \f[C] 11162 --addr string IPaddress:Port or :Port to bind server to (default \[dq]localhost:2121\[dq]) 11163 --auth-proxy string A program to use to create the backend from the auth 11164 --cert string TLS PEM key (concatenation of certificate and CA certificate) 11165 --dir-cache-time Duration Time to cache directory entries for (default 5m0s) 11166 --dir-perms FileMode Directory permissions (default 0777) 11167 --file-perms FileMode File permissions (default 0666) 11168 --gid uint32 Override the gid field set by the filesystem (not supported on Windows) (default 1000) 11169 -h, --help help for ftp 11170 --key string TLS PEM Private key 11171 --no-checksum Don\[aq]t compare checksums on up/download 11172 --no-modtime Don\[aq]t read/write the modification time (can speed things up) 11173 --no-seek Don\[aq]t allow seeking in files 11174 --pass string Password for authentication (empty value allow every password) 11175 --passive-port string Passive port range to use (default \[dq]30000-32000\[dq]) 11176 --poll-interval Duration Time to wait between polling for changes, must be smaller than dir-cache-time and only on supported remotes (set 0 to disable) (default 1m0s) 11177 --public-ip string Public IP address to advertise for passive connections 11178 --read-only Only allow read-only access 11179 --uid uint32 Override the uid field set by the filesystem (not supported on Windows) (default 1000) 11180 --umask int Override the permission bits set by the filesystem (not supported on Windows) (default 2) 11181 --user string User name for authentication (default \[dq]anonymous\[dq]) 11182 --vfs-block-norm-dupes If duplicate filenames exist in the same directory (after normalization), log an error and hide the duplicates (may have a performance cost) 11183 --vfs-cache-max-age Duration Max time since last access of objects in the cache (default 1h0m0s) 11184 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache (default off) 11185 --vfs-cache-min-free-space SizeSuffix Target minimum free space on the disk containing the cache (default off) 11186 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off) 11187 --vfs-cache-poll-interval Duration Interval to poll the cache for stale objects (default 1m0s) 11188 --vfs-case-insensitive If a file name not found, find a case insensitive match 11189 --vfs-disk-space-total-size SizeSuffix Specify the total space of disk (default off) 11190 --vfs-fast-fingerprint Use fast (less accurate) fingerprints for change detection 11191 --vfs-read-ahead SizeSuffix Extra read ahead over --buffer-size when using cache-mode full 11192 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks (default 128Mi) 11193 --vfs-read-chunk-size-limit SizeSuffix If greater than --vfs-read-chunk-size, double the chunk size after each chunk read, until the limit is reached (\[aq]off\[aq] is unlimited) (default off) 11194 --vfs-read-wait Duration Time to wait for in-sequence read before seeking (default 20ms) 11195 --vfs-refresh Refreshes the directory cache recursively in the background on start 11196 --vfs-used-is-size rclone size Use the rclone size algorithm for Used size 11197 --vfs-write-back Duration Time to writeback files after last use when using cache (default 5s) 11198 --vfs-write-wait Duration Time to wait for in-sequence write before giving error (default 1s) 11199 \f[R] 11200 .fi 11201 .SS Filter Options 11202 .PP 11203 Flags for filtering directory listings. 11204 .IP 11205 .nf 11206 \f[C] 11207 --delete-excluded Delete files on dest excluded from sync 11208 --exclude stringArray Exclude files matching pattern 11209 --exclude-from stringArray Read file exclude patterns from file (use - to read from stdin) 11210 --exclude-if-present stringArray Exclude directories if filename is present 11211 --files-from stringArray Read list of source-file names from file (use - to read from stdin) 11212 --files-from-raw stringArray Read list of source-file names from file without any processing of lines (use - to read from stdin) 11213 -f, --filter stringArray Add a file filtering rule 11214 --filter-from stringArray Read file filtering patterns from a file (use - to read from stdin) 11215 --ignore-case Ignore case in filters (case insensitive) 11216 --include stringArray Include files matching pattern 11217 --include-from stringArray Read file include patterns from file (use - to read from stdin) 11218 --max-age Duration Only transfer files younger than this in s or suffix ms|s|m|h|d|w|M|y (default off) 11219 --max-depth int If set limits the recursion depth to this (default -1) 11220 --max-size SizeSuffix Only transfer files smaller than this in KiB or suffix B|K|M|G|T|P (default off) 11221 --metadata-exclude stringArray Exclude metadatas matching pattern 11222 --metadata-exclude-from stringArray Read metadata exclude patterns from file (use - to read from stdin) 11223 --metadata-filter stringArray Add a metadata filtering rule 11224 --metadata-filter-from stringArray Read metadata filtering patterns from a file (use - to read from stdin) 11225 --metadata-include stringArray Include metadatas matching pattern 11226 --metadata-include-from stringArray Read metadata include patterns from file (use - to read from stdin) 11227 --min-age Duration Only transfer files older than this in s or suffix ms|s|m|h|d|w|M|y (default off) 11228 --min-size SizeSuffix Only transfer files bigger than this in KiB or suffix B|K|M|G|T|P (default off) 11229 \f[R] 11230 .fi 11231 .PP 11232 See the global flags page (https://rclone.org/flags/) for global options 11233 not listed here. 11234 .SH SEE ALSO 11235 .IP \[bu] 2 11236 rclone serve (https://rclone.org/commands/rclone_serve/) - Serve a 11237 remote over a protocol. 11238 .SH rclone serve http 11239 .PP 11240 Serve the remote over HTTP. 11241 .SS Synopsis 11242 .PP 11243 Run a basic web server to serve a remote over HTTP. 11244 This can be viewed in a web browser or you can make a remote of type 11245 http read from it. 11246 .PP 11247 You can use the filter flags (e.g. 11248 \f[C]--include\f[R], \f[C]--exclude\f[R]) to control what is served. 11249 .PP 11250 The server will log errors. 11251 Use \f[C]-v\f[R] to see access logs. 11252 .PP 11253 \f[C]--bwlimit\f[R] will be respected for file transfers. 11254 Use \f[C]--stats\f[R] to control the stats printing. 11255 .SS Server options 11256 .PP 11257 Use \f[C]--addr\f[R] to specify which IP address and port the server 11258 should listen on, eg \f[C]--addr 1.2.3.4:8000\f[R] or 11259 \f[C]--addr :8080\f[R] to listen to all IPs. 11260 By default it only listens on localhost. 11261 You can use port :0 to let the OS choose an available port. 11262 .PP 11263 If you set \f[C]--addr\f[R] to listen on a public or LAN accessible IP 11264 address then using Authentication is advised - see the next section for 11265 info. 11266 .PP 11267 You can use a unix socket by setting the url to 11268 \f[C]unix:///path/to/socket\f[R] or just by using an absolute path name. 11269 Note that unix sockets bypass the authentication - this is expected to 11270 be done with file system permissions. 11271 .PP 11272 \f[C]--addr\f[R] may be repeated to listen on multiple 11273 IPs/ports/sockets. 11274 .PP 11275 \f[C]--server-read-timeout\f[R] and \f[C]--server-write-timeout\f[R] can 11276 be used to control the timeouts on the server. 11277 Note that this is the total time for a transfer. 11278 .PP 11279 \f[C]--max-header-bytes\f[R] controls the maximum number of bytes the 11280 server will accept in the HTTP header. 11281 .PP 11282 \f[C]--baseurl\f[R] controls the URL prefix that rclone serves from. 11283 By default rclone will serve from the root. 11284 If you used \f[C]--baseurl \[dq]/rclone\[dq]\f[R] then rclone would 11285 serve from a URL starting with \[dq]/rclone/\[dq]. 11286 This is useful if you wish to proxy rclone serve. 11287 Rclone automatically inserts leading and trailing \[dq]/\[dq] on 11288 \f[C]--baseurl\f[R], so \f[C]--baseurl \[dq]rclone\[dq]\f[R], 11289 \f[C]--baseurl \[dq]/rclone\[dq]\f[R] and 11290 \f[C]--baseurl \[dq]/rclone/\[dq]\f[R] are all treated identically. 11291 .SS TLS (SSL) 11292 .PP 11293 By default this will serve over http. 11294 If you want you can serve over https. 11295 You will need to supply the \f[C]--cert\f[R] and \f[C]--key\f[R] flags. 11296 If you wish to do client side certificate validation then you will need 11297 to supply \f[C]--client-ca\f[R] also. 11298 .PP 11299 \f[C]--cert\f[R] should be a either a PEM encoded certificate or a 11300 concatenation of that with the CA certificate. 11301 \f[C]--key\f[R] should be the PEM encoded private key and 11302 \f[C]--client-ca\f[R] should be the PEM encoded client certificate 11303 authority certificate. 11304 .PP 11305 --min-tls-version is minimum TLS version that is acceptable. 11306 Valid values are \[dq]tls1.0\[dq], \[dq]tls1.1\[dq], \[dq]tls1.2\[dq] 11307 and \[dq]tls1.3\[dq] (default \[dq]tls1.0\[dq]). 11308 .SS Template 11309 .PP 11310 \f[C]--template\f[R] allows a user to specify a custom markup template 11311 for HTTP and WebDAV serve functions. 11312 The server exports the following markup to be used within the template 11313 to server pages: 11314 .PP 11315 .TS 11316 tab(@); 11317 lw(35.0n) lw(35.0n). 11318 T{ 11319 Parameter 11320 T}@T{ 11321 Description 11322 T} 11323 _ 11324 T{ 11325 \&.Name 11326 T}@T{ 11327 The full path of a file/directory. 11328 T} 11329 T{ 11330 \&.Title 11331 T}@T{ 11332 Directory listing of .Name 11333 T} 11334 T{ 11335 \&.Sort 11336 T}@T{ 11337 The current sort used. 11338 This is changeable via ?sort= parameter 11339 T} 11340 T{ 11341 T}@T{ 11342 Sort Options: namedirfirst,name,size,time (default namedirfirst) 11343 T} 11344 T{ 11345 \&.Order 11346 T}@T{ 11347 The current ordering used. 11348 This is changeable via ?order= parameter 11349 T} 11350 T{ 11351 T}@T{ 11352 Order Options: asc,desc (default asc) 11353 T} 11354 T{ 11355 \&.Query 11356 T}@T{ 11357 Currently unused. 11358 T} 11359 T{ 11360 \&.Breadcrumb 11361 T}@T{ 11362 Allows for creating a relative navigation 11363 T} 11364 T{ 11365 -- .Link 11366 T}@T{ 11367 The relative to the root link of the Text. 11368 T} 11369 T{ 11370 -- .Text 11371 T}@T{ 11372 The Name of the directory. 11373 T} 11374 T{ 11375 \&.Entries 11376 T}@T{ 11377 Information about a specific file/directory. 11378 T} 11379 T{ 11380 -- .URL 11381 T}@T{ 11382 The \[aq]url\[aq] of an entry. 11383 T} 11384 T{ 11385 -- .Leaf 11386 T}@T{ 11387 Currently same as \[aq]URL\[aq] but intended to be \[aq]just\[aq] the 11388 name. 11389 T} 11390 T{ 11391 -- .IsDir 11392 T}@T{ 11393 Boolean for if an entry is a directory or not. 11394 T} 11395 T{ 11396 -- .Size 11397 T}@T{ 11398 Size in Bytes of the entry. 11399 T} 11400 T{ 11401 -- .ModTime 11402 T}@T{ 11403 The UTC timestamp of an entry. 11404 T} 11405 .TE 11406 .PP 11407 The server also makes the following functions available so that they can 11408 be used within the template. 11409 These functions help extend the options for dynamic rendering of HTML. 11410 They can be used to render HTML based on specific conditions. 11411 .PP 11412 .TS 11413 tab(@); 11414 lw(35.0n) lw(35.0n). 11415 T{ 11416 Function 11417 T}@T{ 11418 Description 11419 T} 11420 _ 11421 T{ 11422 afterEpoch 11423 T}@T{ 11424 Returns the time since the epoch for the given time. 11425 T} 11426 T{ 11427 contains 11428 T}@T{ 11429 Checks whether a given substring is present or not in a given string. 11430 T} 11431 T{ 11432 hasPrefix 11433 T}@T{ 11434 Checks whether the given string begins with the specified prefix. 11435 T} 11436 T{ 11437 hasSuffix 11438 T}@T{ 11439 Checks whether the given string end with the specified suffix. 11440 T} 11441 .TE 11442 .SS Authentication 11443 .PP 11444 By default this will serve files without needing a login. 11445 .PP 11446 You can either use an htpasswd file which can take lots of users, or set 11447 a single username and password with the \f[C]--user\f[R] and 11448 \f[C]--pass\f[R] flags. 11449 .PP 11450 If no static users are configured by either of the above methods, and 11451 client certificates are required by the \f[C]--client-ca\f[R] flag 11452 passed to the server, the client certificate common name will be 11453 considered as the username. 11454 .PP 11455 Use \f[C]--htpasswd /path/to/htpasswd\f[R] to provide an htpasswd file. 11456 This is in standard apache format and supports MD5, SHA1 and BCrypt for 11457 basic authentication. 11458 Bcrypt is recommended. 11459 .PP 11460 To create an htpasswd file: 11461 .IP 11462 .nf 11463 \f[C] 11464 touch htpasswd 11465 htpasswd -B htpasswd user 11466 htpasswd -B htpasswd anotherUser 11467 \f[R] 11468 .fi 11469 .PP 11470 The password file can be updated while rclone is running. 11471 .PP 11472 Use \f[C]--realm\f[R] to set the authentication realm. 11473 .PP 11474 Use \f[C]--salt\f[R] to change the password hashing salt from the 11475 default. 11476 ## VFS - Virtual File System 11477 .PP 11478 This command uses the VFS layer. 11479 This adapts the cloud storage objects that rclone uses into something 11480 which looks much more like a disk filing system. 11481 .PP 11482 Cloud storage objects have lots of properties which aren\[aq]t like disk 11483 files - you can\[aq]t extend them or write to the middle of them, so the 11484 VFS layer has to deal with that. 11485 Because there is no one right way of doing this there are various 11486 options explained below. 11487 .PP 11488 The VFS layer also implements a directory cache - this caches info about 11489 files and directories (but not the data) in memory. 11490 .SS VFS Directory Cache 11491 .PP 11492 Using the \f[C]--dir-cache-time\f[R] flag, you can control how long a 11493 directory should be considered up to date and not refreshed from the 11494 backend. 11495 Changes made through the VFS will appear immediately or invalidate the 11496 cache. 11497 .IP 11498 .nf 11499 \f[C] 11500 --dir-cache-time duration Time to cache directory entries for (default 5m0s) 11501 --poll-interval duration Time to wait between polling for changes. Must be smaller than dir-cache-time. Only on supported remotes. Set to 0 to disable (default 1m0s) 11502 \f[R] 11503 .fi 11504 .PP 11505 However, changes made directly on the cloud storage by the web interface 11506 or a different copy of rclone will only be picked up once the directory 11507 cache expires if the backend configured does not support polling for 11508 changes. 11509 If the backend supports polling, changes will be picked up within the 11510 polling interval. 11511 .PP 11512 You can send a \f[C]SIGHUP\f[R] signal to rclone for it to flush all 11513 directory caches, regardless of how old they are. 11514 Assuming only one rclone instance is running, you can reset the cache 11515 like this: 11516 .IP 11517 .nf 11518 \f[C] 11519 kill -SIGHUP $(pidof rclone) 11520 \f[R] 11521 .fi 11522 .PP 11523 If you configure rclone with a remote control then you can use rclone rc 11524 to flush the whole directory cache: 11525 .IP 11526 .nf 11527 \f[C] 11528 rclone rc vfs/forget 11529 \f[R] 11530 .fi 11531 .PP 11532 Or individual files or directories: 11533 .IP 11534 .nf 11535 \f[C] 11536 rclone rc vfs/forget file=path/to/file dir=path/to/dir 11537 \f[R] 11538 .fi 11539 .SS VFS File Buffering 11540 .PP 11541 The \f[C]--buffer-size\f[R] flag determines the amount of memory, that 11542 will be used to buffer data in advance. 11543 .PP 11544 Each open file will try to keep the specified amount of data in memory 11545 at all times. 11546 The buffered data is bound to one open file and won\[aq]t be shared. 11547 .PP 11548 This flag is a upper limit for the used memory per open file. 11549 The buffer will only use memory for data that is downloaded but not not 11550 yet read. 11551 If the buffer is empty, only a small amount of memory will be used. 11552 .PP 11553 The maximum memory used by rclone for buffering can be up to 11554 \f[C]--buffer-size * open files\f[R]. 11555 .SS VFS File Caching 11556 .PP 11557 These flags control the VFS file caching options. 11558 File caching is necessary to make the VFS layer appear compatible with a 11559 normal file system. 11560 It can be disabled at the cost of some compatibility. 11561 .PP 11562 For example you\[aq]ll need to enable VFS caching if you want to read 11563 and write simultaneously to a file. 11564 See below for more details. 11565 .PP 11566 Note that the VFS cache is separate from the cache backend and you may 11567 find that you need one or the other or both. 11568 .IP 11569 .nf 11570 \f[C] 11571 --cache-dir string Directory rclone will use for caching. 11572 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off) 11573 --vfs-cache-max-age duration Max time since last access of objects in the cache (default 1h0m0s) 11574 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache (default off) 11575 --vfs-cache-min-free-space SizeSuffix Target minimum free space on the disk containing the cache (default off) 11576 --vfs-cache-poll-interval duration Interval to poll the cache for stale objects (default 1m0s) 11577 --vfs-write-back duration Time to writeback files after last use when using cache (default 5s) 11578 \f[R] 11579 .fi 11580 .PP 11581 If run with \f[C]-vv\f[R] rclone will print the location of the file 11582 cache. 11583 The files are stored in the user cache file area which is OS dependent 11584 but can be controlled with \f[C]--cache-dir\f[R] or setting the 11585 appropriate environment variable. 11586 .PP 11587 The cache has 4 different modes selected by \f[C]--vfs-cache-mode\f[R]. 11588 The higher the cache mode the more compatible rclone becomes at the cost 11589 of using disk space. 11590 .PP 11591 Note that files are written back to the remote only when they are closed 11592 and if they haven\[aq]t been accessed for \f[C]--vfs-write-back\f[R] 11593 seconds. 11594 If rclone is quit or dies with files that haven\[aq]t been uploaded, 11595 these will be uploaded next time rclone is run with the same flags. 11596 .PP 11597 If using \f[C]--vfs-cache-max-size\f[R] or 11598 \f[C]--vfs-cache-min-free-size\f[R] note that the cache may exceed these 11599 quotas for two reasons. 11600 Firstly because it is only checked every 11601 \f[C]--vfs-cache-poll-interval\f[R]. 11602 Secondly because open files cannot be evicted from the cache. 11603 When \f[C]--vfs-cache-max-size\f[R] or 11604 \f[C]--vfs-cache-min-free-size\f[R] is exceeded, rclone will attempt to 11605 evict the least accessed files from the cache first. 11606 rclone will start with files that haven\[aq]t been accessed for the 11607 longest. 11608 This cache flushing strategy is efficient and more relevant files are 11609 likely to remain cached. 11610 .PP 11611 The \f[C]--vfs-cache-max-age\f[R] will evict files from the cache after 11612 the set time since last access has passed. 11613 The default value of 1 hour will start evicting files from cache that 11614 haven\[aq]t been accessed for 1 hour. 11615 When a cached file is accessed the 1 hour timer is reset to 0 and will 11616 wait for 1 more hour before evicting. 11617 Specify the time with standard notation, s, m, h, d, w . 11618 .PP 11619 You \f[B]should not\f[R] run two copies of rclone using the same VFS 11620 cache with the same or overlapping remotes if using 11621 \f[C]--vfs-cache-mode > off\f[R]. 11622 This can potentially cause data corruption if you do. 11623 You can work around this by giving each rclone its own cache hierarchy 11624 with \f[C]--cache-dir\f[R]. 11625 You don\[aq]t need to worry about this if the remotes in use don\[aq]t 11626 overlap. 11627 .SS --vfs-cache-mode off 11628 .PP 11629 In this mode (the default) the cache will read directly from the remote 11630 and write directly to the remote without caching anything on disk. 11631 .PP 11632 This will mean some operations are not possible 11633 .IP \[bu] 2 11634 Files can\[aq]t be opened for both read AND write 11635 .IP \[bu] 2 11636 Files opened for write can\[aq]t be seeked 11637 .IP \[bu] 2 11638 Existing files opened for write must have O_TRUNC set 11639 .IP \[bu] 2 11640 Files open for read with O_TRUNC will be opened write only 11641 .IP \[bu] 2 11642 Files open for write only will behave as if O_TRUNC was supplied 11643 .IP \[bu] 2 11644 Open modes O_APPEND, O_TRUNC are ignored 11645 .IP \[bu] 2 11646 If an upload fails it can\[aq]t be retried 11647 .SS --vfs-cache-mode minimal 11648 .PP 11649 This is very similar to \[dq]off\[dq] except that files opened for read 11650 AND write will be buffered to disk. 11651 This means that files opened for write will be a lot more compatible, 11652 but uses the minimal disk space. 11653 .PP 11654 These operations are not possible 11655 .IP \[bu] 2 11656 Files opened for write only can\[aq]t be seeked 11657 .IP \[bu] 2 11658 Existing files opened for write must have O_TRUNC set 11659 .IP \[bu] 2 11660 Files opened for write only will ignore O_APPEND, O_TRUNC 11661 .IP \[bu] 2 11662 If an upload fails it can\[aq]t be retried 11663 .SS --vfs-cache-mode writes 11664 .PP 11665 In this mode files opened for read only are still read directly from the 11666 remote, write only and read/write files are buffered to disk first. 11667 .PP 11668 This mode should support all normal file system operations. 11669 .PP 11670 If an upload fails it will be retried at exponentially increasing 11671 intervals up to 1 minute. 11672 .SS --vfs-cache-mode full 11673 .PP 11674 In this mode all reads and writes are buffered to and from disk. 11675 When data is read from the remote this is buffered to disk as well. 11676 .PP 11677 In this mode the files in the cache will be sparse files and rclone will 11678 keep track of which bits of the files it has downloaded. 11679 .PP 11680 So if an application only reads the starts of each file, then rclone 11681 will only buffer the start of the file. 11682 These files will appear to be their full size in the cache, but they 11683 will be sparse files with only the data that has been downloaded present 11684 in them. 11685 .PP 11686 This mode should support all normal file system operations and is 11687 otherwise identical to \f[C]--vfs-cache-mode\f[R] writes. 11688 .PP 11689 When reading a file rclone will read \f[C]--buffer-size\f[R] plus 11690 \f[C]--vfs-read-ahead\f[R] bytes ahead. 11691 The \f[C]--buffer-size\f[R] is buffered in memory whereas the 11692 \f[C]--vfs-read-ahead\f[R] is buffered on disk. 11693 .PP 11694 When using this mode it is recommended that \f[C]--buffer-size\f[R] is 11695 not set too large and \f[C]--vfs-read-ahead\f[R] is set large if 11696 required. 11697 .PP 11698 \f[B]IMPORTANT\f[R] not all file systems support sparse files. 11699 In particular FAT/exFAT do not. 11700 Rclone will perform very badly if the cache directory is on a filesystem 11701 which doesn\[aq]t support sparse files and it will log an ERROR message 11702 if one is detected. 11703 .SS Fingerprinting 11704 .PP 11705 Various parts of the VFS use fingerprinting to see if a local file copy 11706 has changed relative to a remote file. 11707 Fingerprints are made from: 11708 .IP \[bu] 2 11709 size 11710 .IP \[bu] 2 11711 modification time 11712 .IP \[bu] 2 11713 hash 11714 .PP 11715 where available on an object. 11716 .PP 11717 On some backends some of these attributes are slow to read (they take an 11718 extra API call per object, or extra work per object). 11719 .PP 11720 For example \f[C]hash\f[R] is slow with the \f[C]local\f[R] and 11721 \f[C]sftp\f[R] backends as they have to read the entire file and hash 11722 it, and \f[C]modtime\f[R] is slow with the \f[C]s3\f[R], 11723 \f[C]swift\f[R], \f[C]ftp\f[R] and \f[C]qinqstor\f[R] backends because 11724 they need to do an extra API call to fetch it. 11725 .PP 11726 If you use the \f[C]--vfs-fast-fingerprint\f[R] flag then rclone will 11727 not include the slow operations in the fingerprint. 11728 This makes the fingerprinting less accurate but much faster and will 11729 improve the opening time of cached files. 11730 .PP 11731 If you are running a vfs cache over \f[C]local\f[R], \f[C]s3\f[R] or 11732 \f[C]swift\f[R] backends then using this flag is recommended. 11733 .PP 11734 Note that if you change the value of this flag, the fingerprints of the 11735 files in the cache may be invalidated and the files will need to be 11736 downloaded again. 11737 .SS VFS Chunked Reading 11738 .PP 11739 When rclone reads files from a remote it reads them in chunks. 11740 This means that rather than requesting the whole file rclone reads the 11741 chunk specified. 11742 This can reduce the used download quota for some remotes by requesting 11743 only chunks from the remote that are actually read, at the cost of an 11744 increased number of requests. 11745 .PP 11746 These flags control the chunking: 11747 .IP 11748 .nf 11749 \f[C] 11750 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks (default 128M) 11751 --vfs-read-chunk-size-limit SizeSuffix Max chunk doubling size (default off) 11752 \f[R] 11753 .fi 11754 .PP 11755 Rclone will start reading a chunk of size 11756 \f[C]--vfs-read-chunk-size\f[R], and then double the size for each read. 11757 When \f[C]--vfs-read-chunk-size-limit\f[R] is specified, and greater 11758 than \f[C]--vfs-read-chunk-size\f[R], the chunk size for each open file 11759 will get doubled only until the specified value is reached. 11760 If the value is \[dq]off\[dq], which is the default, the limit is 11761 disabled and the chunk size will grow indefinitely. 11762 .PP 11763 With \f[C]--vfs-read-chunk-size 100M\f[R] and 11764 \f[C]--vfs-read-chunk-size-limit 0\f[R] the following parts will be 11765 downloaded: 0-100M, 100M-200M, 200M-300M, 300M-400M and so on. 11766 When \f[C]--vfs-read-chunk-size-limit 500M\f[R] is specified, the result 11767 would be 0-100M, 100M-300M, 300M-700M, 700M-1200M, 1200M-1700M and so 11768 on. 11769 .PP 11770 Setting \f[C]--vfs-read-chunk-size\f[R] to \f[C]0\f[R] or \[dq]off\[dq] 11771 disables chunked reading. 11772 .SS VFS Performance 11773 .PP 11774 These flags may be used to enable/disable features of the VFS for 11775 performance or other reasons. 11776 See also the chunked reading feature. 11777 .PP 11778 In particular S3 and Swift benefit hugely from the 11779 \f[C]--no-modtime\f[R] flag (or use \f[C]--use-server-modtime\f[R] for a 11780 slightly different effect) as each read of the modification time takes a 11781 transaction. 11782 .IP 11783 .nf 11784 \f[C] 11785 --no-checksum Don\[aq]t compare checksums on up/download. 11786 --no-modtime Don\[aq]t read/write the modification time (can speed things up). 11787 --no-seek Don\[aq]t allow seeking in files. 11788 --read-only Only allow read-only access. 11789 \f[R] 11790 .fi 11791 .PP 11792 Sometimes rclone is delivered reads or writes out of order. 11793 Rather than seeking rclone will wait a short time for the in sequence 11794 read or write to come in. 11795 These flags only come into effect when not using an on disk cache file. 11796 .IP 11797 .nf 11798 \f[C] 11799 --vfs-read-wait duration Time to wait for in-sequence read before seeking (default 20ms) 11800 --vfs-write-wait duration Time to wait for in-sequence write before giving error (default 1s) 11801 \f[R] 11802 .fi 11803 .PP 11804 When using VFS write caching (\f[C]--vfs-cache-mode\f[R] with value 11805 writes or full), the global flag \f[C]--transfers\f[R] can be set to 11806 adjust the number of parallel uploads of modified files from the cache 11807 (the related global flag \f[C]--checkers\f[R] has no effect on the VFS). 11808 .IP 11809 .nf 11810 \f[C] 11811 --transfers int Number of file transfers to run in parallel (default 4) 11812 \f[R] 11813 .fi 11814 .SS VFS Case Sensitivity 11815 .PP 11816 Linux file systems are case-sensitive: two files can differ only by 11817 case, and the exact case must be used when opening a file. 11818 .PP 11819 File systems in modern Windows are case-insensitive but case-preserving: 11820 although existing files can be opened using any case, the exact case 11821 used to create the file is preserved and available for programs to 11822 query. 11823 It is not allowed for two files in the same directory to differ only by 11824 case. 11825 .PP 11826 Usually file systems on macOS are case-insensitive. 11827 It is possible to make macOS file systems case-sensitive but that is not 11828 the default. 11829 .PP 11830 The \f[C]--vfs-case-insensitive\f[R] VFS flag controls how rclone 11831 handles these two cases. 11832 If its value is \[dq]false\[dq], rclone passes file names to the remote 11833 as-is. 11834 If the flag is \[dq]true\[dq] (or appears without a value on the command 11835 line), rclone may perform a \[dq]fixup\[dq] as explained below. 11836 .PP 11837 The user may specify a file name to open/delete/rename/etc with a case 11838 different than what is stored on the remote. 11839 If an argument refers to an existing file with exactly the same name, 11840 then the case of the existing file on the disk will be used. 11841 However, if a file name with exactly the same name is not found but a 11842 name differing only by case exists, rclone will transparently fixup the 11843 name. 11844 This fixup happens only when an existing file is requested. 11845 Case sensitivity of file names created anew by rclone is controlled by 11846 the underlying remote. 11847 .PP 11848 Note that case sensitivity of the operating system running rclone (the 11849 target) may differ from case sensitivity of a file system presented by 11850 rclone (the source). 11851 The flag controls whether \[dq]fixup\[dq] is performed to satisfy the 11852 target. 11853 .PP 11854 If the flag is not provided on the command line, then its default value 11855 depends on the operating system where rclone runs: \[dq]true\[dq] on 11856 Windows and macOS, \[dq]false\[dq] otherwise. 11857 If the flag is provided without a value, then it is \[dq]true\[dq]. 11858 .PP 11859 The \f[C]--no-unicode-normalization\f[R] flag controls whether a similar 11860 \[dq]fixup\[dq] is performed for filenames that differ but are 11861 canonically 11862 equivalent (https://en.wikipedia.org/wiki/Unicode_equivalence) with 11863 respect to unicode. 11864 Unicode normalization can be particularly helpful for users of macOS, 11865 which prefers form NFD instead of the NFC used by most other platforms. 11866 It is therefore highly recommended to keep the default of 11867 \f[C]false\f[R] on macOS, to avoid encoding compatibility issues. 11868 .PP 11869 In the (probably unlikely) event that a directory has multiple duplicate 11870 filenames after applying case and unicode normalization, the 11871 \f[C]--vfs-block-norm-dupes\f[R] flag allows hiding these duplicates. 11872 This comes with a performance tradeoff, as rclone will have to scan the 11873 entire directory for duplicates when listing a directory. 11874 For this reason, it is recommended to leave this disabled if not needed. 11875 However, macOS users may wish to consider using it, as otherwise, if a 11876 remote directory contains both NFC and NFD versions of the same 11877 filename, an odd situation will occur: both versions of the file will be 11878 visible in the mount, and both will appear to be editable, however, 11879 editing either version will actually result in only the NFD version 11880 getting edited under the hood. 11881 \f[C]--vfs-block- norm-dupes\f[R] prevents this confusion by detecting 11882 this scenario, hiding the duplicates, and logging an error, similar to 11883 how this is handled in \f[C]rclone sync\f[R]. 11884 .SS VFS Disk Options 11885 .PP 11886 This flag allows you to manually set the statistics about the filing 11887 system. 11888 It can be useful when those statistics cannot be read correctly 11889 automatically. 11890 .IP 11891 .nf 11892 \f[C] 11893 --vfs-disk-space-total-size Manually set the total disk space size (example: 256G, default: -1) 11894 \f[R] 11895 .fi 11896 .SS Alternate report of used bytes 11897 .PP 11898 Some backends, most notably S3, do not report the amount of bytes used. 11899 If you need this information to be available when running \f[C]df\f[R] 11900 on the filesystem, then pass the flag \f[C]--vfs-used-is-size\f[R] to 11901 rclone. 11902 With this flag set, instead of relying on the backend to report this 11903 information, rclone will scan the whole remote similar to 11904 \f[C]rclone size\f[R] and compute the total used space itself. 11905 .PP 11906 \f[I]WARNING.\f[R] Contrary to \f[C]rclone size\f[R], this flag ignores 11907 filters so that the result is accurate. 11908 However, this is very inefficient and may cost lots of API calls 11909 resulting in extra charges. 11910 Use it as a last resort and only with caching. 11911 .SS Auth Proxy 11912 .PP 11913 If you supply the parameter \f[C]--auth-proxy /path/to/program\f[R] then 11914 rclone will use that program to generate backends on the fly which then 11915 are used to authenticate incoming requests. 11916 This uses a simple JSON based protocol with input on STDIN and output on 11917 STDOUT. 11918 .PP 11919 \f[B]PLEASE NOTE:\f[R] \f[C]--auth-proxy\f[R] and 11920 \f[C]--authorized-keys\f[R] cannot be used together, if 11921 \f[C]--auth-proxy\f[R] is set the authorized keys option will be 11922 ignored. 11923 .PP 11924 There is an example program 11925 bin/test_proxy.py (https://github.com/artpar/artpar/blob/master/bin/test_proxy.py) 11926 in the rclone source code. 11927 .PP 11928 The program\[aq]s job is to take a \f[C]user\f[R] and \f[C]pass\f[R] on 11929 the input and turn those into the config for a backend on STDOUT in JSON 11930 format. 11931 This config will have any default parameters for the backend added, but 11932 it won\[aq]t use configuration from environment variables or command 11933 line options - it is the job of the proxy program to make a complete 11934 config. 11935 .PP 11936 This config generated must have this extra parameter - \f[C]_root\f[R] - 11937 root to use for the backend 11938 .PP 11939 And it may have this parameter - \f[C]_obscure\f[R] - comma separated 11940 strings for parameters to obscure 11941 .PP 11942 If password authentication was used by the client, input to the proxy 11943 process (on STDIN) would look similar to this: 11944 .IP 11945 .nf 11946 \f[C] 11947 { 11948 \[dq]user\[dq]: \[dq]me\[dq], 11949 \[dq]pass\[dq]: \[dq]mypassword\[dq] 11950 } 11951 \f[R] 11952 .fi 11953 .PP 11954 If public-key authentication was used by the client, input to the proxy 11955 process (on STDIN) would look similar to this: 11956 .IP 11957 .nf 11958 \f[C] 11959 { 11960 \[dq]user\[dq]: \[dq]me\[dq], 11961 \[dq]public_key\[dq]: \[dq]AAAAB3NzaC1yc2EAAAADAQABAAABAQDuwESFdAe14hVS6omeyX7edc...JQdf\[dq] 11962 } 11963 \f[R] 11964 .fi 11965 .PP 11966 And as an example return this on STDOUT 11967 .IP 11968 .nf 11969 \f[C] 11970 { 11971 \[dq]type\[dq]: \[dq]sftp\[dq], 11972 \[dq]_root\[dq]: \[dq]\[dq], 11973 \[dq]_obscure\[dq]: \[dq]pass\[dq], 11974 \[dq]user\[dq]: \[dq]me\[dq], 11975 \[dq]pass\[dq]: \[dq]mypassword\[dq], 11976 \[dq]host\[dq]: \[dq]sftp.example.com\[dq] 11977 } 11978 \f[R] 11979 .fi 11980 .PP 11981 This would mean that an SFTP backend would be created on the fly for the 11982 \f[C]user\f[R] and \f[C]pass\f[R]/\f[C]public_key\f[R] returned in the 11983 output to the host given. 11984 Note that since \f[C]_obscure\f[R] is set to \f[C]pass\f[R], rclone will 11985 obscure the \f[C]pass\f[R] parameter before creating the backend (which 11986 is required for sftp backends). 11987 .PP 11988 The program can manipulate the supplied \f[C]user\f[R] in any way, for 11989 example to make proxy to many different sftp backends, you could make 11990 the \f[C]user\f[R] be \f[C]user\[at]example.com\f[R] and then set the 11991 \f[C]host\f[R] to \f[C]example.com\f[R] in the output and the user to 11992 \f[C]user\f[R]. 11993 For security you\[aq]d probably want to restrict the \f[C]host\f[R] to a 11994 limited list. 11995 .PP 11996 Note that an internal cache is keyed on \f[C]user\f[R] so only use that 11997 for configuration, don\[aq]t use \f[C]pass\f[R] or \f[C]public_key\f[R]. 11998 This also means that if a user\[aq]s password or public-key is changed 11999 the cache will need to expire (which takes 5 mins) before it takes 12000 effect. 12001 .PP 12002 This can be used to build general purpose proxies to any kind of backend 12003 that rclone supports. 12004 .IP 12005 .nf 12006 \f[C] 12007 rclone serve http remote:path [flags] 12008 \f[R] 12009 .fi 12010 .SS Options 12011 .IP 12012 .nf 12013 \f[C] 12014 --addr stringArray IPaddress:Port or :Port to bind server to (default [127.0.0.1:8080]) 12015 --allow-origin string Origin which cross-domain request (CORS) can be executed from 12016 --auth-proxy string A program to use to create the backend from the auth 12017 --baseurl string Prefix for URLs - leave blank for root 12018 --cert string TLS PEM key (concatenation of certificate and CA certificate) 12019 --client-ca string Client certificate authority to verify clients with 12020 --dir-cache-time Duration Time to cache directory entries for (default 5m0s) 12021 --dir-perms FileMode Directory permissions (default 0777) 12022 --file-perms FileMode File permissions (default 0666) 12023 --gid uint32 Override the gid field set by the filesystem (not supported on Windows) (default 1000) 12024 -h, --help help for http 12025 --htpasswd string A htpasswd file - if not provided no authentication is done 12026 --key string TLS PEM Private key 12027 --max-header-bytes int Maximum size of request header (default 4096) 12028 --min-tls-version string Minimum TLS version that is acceptable (default \[dq]tls1.0\[dq]) 12029 --no-checksum Don\[aq]t compare checksums on up/download 12030 --no-modtime Don\[aq]t read/write the modification time (can speed things up) 12031 --no-seek Don\[aq]t allow seeking in files 12032 --pass string Password for authentication 12033 --poll-interval Duration Time to wait between polling for changes, must be smaller than dir-cache-time and only on supported remotes (set 0 to disable) (default 1m0s) 12034 --read-only Only allow read-only access 12035 --realm string Realm for authentication 12036 --salt string Password hashing salt (default \[dq]dlPL2MqE\[dq]) 12037 --server-read-timeout Duration Timeout for server reading data (default 1h0m0s) 12038 --server-write-timeout Duration Timeout for server writing data (default 1h0m0s) 12039 --template string User-specified template 12040 --uid uint32 Override the uid field set by the filesystem (not supported on Windows) (default 1000) 12041 --umask int Override the permission bits set by the filesystem (not supported on Windows) (default 2) 12042 --user string User name for authentication 12043 --vfs-block-norm-dupes If duplicate filenames exist in the same directory (after normalization), log an error and hide the duplicates (may have a performance cost) 12044 --vfs-cache-max-age Duration Max time since last access of objects in the cache (default 1h0m0s) 12045 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache (default off) 12046 --vfs-cache-min-free-space SizeSuffix Target minimum free space on the disk containing the cache (default off) 12047 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off) 12048 --vfs-cache-poll-interval Duration Interval to poll the cache for stale objects (default 1m0s) 12049 --vfs-case-insensitive If a file name not found, find a case insensitive match 12050 --vfs-disk-space-total-size SizeSuffix Specify the total space of disk (default off) 12051 --vfs-fast-fingerprint Use fast (less accurate) fingerprints for change detection 12052 --vfs-read-ahead SizeSuffix Extra read ahead over --buffer-size when using cache-mode full 12053 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks (default 128Mi) 12054 --vfs-read-chunk-size-limit SizeSuffix If greater than --vfs-read-chunk-size, double the chunk size after each chunk read, until the limit is reached (\[aq]off\[aq] is unlimited) (default off) 12055 --vfs-read-wait Duration Time to wait for in-sequence read before seeking (default 20ms) 12056 --vfs-refresh Refreshes the directory cache recursively in the background on start 12057 --vfs-used-is-size rclone size Use the rclone size algorithm for Used size 12058 --vfs-write-back Duration Time to writeback files after last use when using cache (default 5s) 12059 --vfs-write-wait Duration Time to wait for in-sequence write before giving error (default 1s) 12060 \f[R] 12061 .fi 12062 .SS Filter Options 12063 .PP 12064 Flags for filtering directory listings. 12065 .IP 12066 .nf 12067 \f[C] 12068 --delete-excluded Delete files on dest excluded from sync 12069 --exclude stringArray Exclude files matching pattern 12070 --exclude-from stringArray Read file exclude patterns from file (use - to read from stdin) 12071 --exclude-if-present stringArray Exclude directories if filename is present 12072 --files-from stringArray Read list of source-file names from file (use - to read from stdin) 12073 --files-from-raw stringArray Read list of source-file names from file without any processing of lines (use - to read from stdin) 12074 -f, --filter stringArray Add a file filtering rule 12075 --filter-from stringArray Read file filtering patterns from a file (use - to read from stdin) 12076 --ignore-case Ignore case in filters (case insensitive) 12077 --include stringArray Include files matching pattern 12078 --include-from stringArray Read file include patterns from file (use - to read from stdin) 12079 --max-age Duration Only transfer files younger than this in s or suffix ms|s|m|h|d|w|M|y (default off) 12080 --max-depth int If set limits the recursion depth to this (default -1) 12081 --max-size SizeSuffix Only transfer files smaller than this in KiB or suffix B|K|M|G|T|P (default off) 12082 --metadata-exclude stringArray Exclude metadatas matching pattern 12083 --metadata-exclude-from stringArray Read metadata exclude patterns from file (use - to read from stdin) 12084 --metadata-filter stringArray Add a metadata filtering rule 12085 --metadata-filter-from stringArray Read metadata filtering patterns from a file (use - to read from stdin) 12086 --metadata-include stringArray Include metadatas matching pattern 12087 --metadata-include-from stringArray Read metadata include patterns from file (use - to read from stdin) 12088 --min-age Duration Only transfer files older than this in s or suffix ms|s|m|h|d|w|M|y (default off) 12089 --min-size SizeSuffix Only transfer files bigger than this in KiB or suffix B|K|M|G|T|P (default off) 12090 \f[R] 12091 .fi 12092 .PP 12093 See the global flags page (https://rclone.org/flags/) for global options 12094 not listed here. 12095 .SH SEE ALSO 12096 .IP \[bu] 2 12097 rclone serve (https://rclone.org/commands/rclone_serve/) - Serve a 12098 remote over a protocol. 12099 .SH rclone serve nfs 12100 .PP 12101 Serve the remote as an NFS mount 12102 .SS Synopsis 12103 .PP 12104 Create an NFS server that serves the given remote over the network. 12105 .PP 12106 The primary purpose for this command is to enable mount 12107 command (https://rclone.org/commands/rclone_mount/) on recent macOS 12108 versions where installing FUSE is very cumbersome. 12109 .PP 12110 Since this is running on NFSv3, no authentication method is available. 12111 Any client will be able to access the data. 12112 To limit access, you can use serve NFS on loopback address and rely on 12113 secure tunnels (such as SSH). 12114 For this reason, by default, a random TCP port is chosen and loopback 12115 interface is used for the listening address; meaning that it is only 12116 available to the local machine. 12117 If you want other machines to access the NFS mount over local network, 12118 you need to specify the listening address and port using 12119 \f[C]--addr\f[R] flag. 12120 .PP 12121 Modifying files through NFS protocol requires VFS caching. 12122 Usually you will need to specify \f[C]--vfs-cache-mode\f[R] in order to 12123 be able to write to the mountpoint (full is recommended). 12124 If you don\[aq]t specify VFS cache mode, the mount will be read-only. 12125 Note also that \f[C]--nfs-cache-handle-limit\f[R] controls the maximum 12126 number of cached file handles stored by the caching handler. 12127 This should not be set too low or you may experience errors when trying 12128 to access files. 12129 The default is \f[C]1000000\f[R], but consider lowering this limit if 12130 the server\[aq]s system resource usage causes problems. 12131 .PP 12132 To serve NFS over the network use following command: 12133 .IP 12134 .nf 12135 \f[C] 12136 rclone serve nfs remote: --addr 0.0.0.0:$PORT --vfs-cache-mode=full 12137 \f[R] 12138 .fi 12139 .PP 12140 We specify a specific port that we can use in the mount command: 12141 .PP 12142 To mount the server under Linux/macOS, use the following command: 12143 .IP 12144 .nf 12145 \f[C] 12146 mount -oport=$PORT,mountport=$PORT $HOSTNAME: path/to/mountpoint 12147 \f[R] 12148 .fi 12149 .PP 12150 Where \f[C]$PORT\f[R] is the same port number we used in the serve nfs 12151 command. 12152 .PP 12153 This feature is only available on Unix platforms. 12154 .SS VFS - Virtual File System 12155 .PP 12156 This command uses the VFS layer. 12157 This adapts the cloud storage objects that rclone uses into something 12158 which looks much more like a disk filing system. 12159 .PP 12160 Cloud storage objects have lots of properties which aren\[aq]t like disk 12161 files - you can\[aq]t extend them or write to the middle of them, so the 12162 VFS layer has to deal with that. 12163 Because there is no one right way of doing this there are various 12164 options explained below. 12165 .PP 12166 The VFS layer also implements a directory cache - this caches info about 12167 files and directories (but not the data) in memory. 12168 .SS VFS Directory Cache 12169 .PP 12170 Using the \f[C]--dir-cache-time\f[R] flag, you can control how long a 12171 directory should be considered up to date and not refreshed from the 12172 backend. 12173 Changes made through the VFS will appear immediately or invalidate the 12174 cache. 12175 .IP 12176 .nf 12177 \f[C] 12178 --dir-cache-time duration Time to cache directory entries for (default 5m0s) 12179 --poll-interval duration Time to wait between polling for changes. Must be smaller than dir-cache-time. Only on supported remotes. Set to 0 to disable (default 1m0s) 12180 \f[R] 12181 .fi 12182 .PP 12183 However, changes made directly on the cloud storage by the web interface 12184 or a different copy of rclone will only be picked up once the directory 12185 cache expires if the backend configured does not support polling for 12186 changes. 12187 If the backend supports polling, changes will be picked up within the 12188 polling interval. 12189 .PP 12190 You can send a \f[C]SIGHUP\f[R] signal to rclone for it to flush all 12191 directory caches, regardless of how old they are. 12192 Assuming only one rclone instance is running, you can reset the cache 12193 like this: 12194 .IP 12195 .nf 12196 \f[C] 12197 kill -SIGHUP $(pidof rclone) 12198 \f[R] 12199 .fi 12200 .PP 12201 If you configure rclone with a remote control then you can use rclone rc 12202 to flush the whole directory cache: 12203 .IP 12204 .nf 12205 \f[C] 12206 rclone rc vfs/forget 12207 \f[R] 12208 .fi 12209 .PP 12210 Or individual files or directories: 12211 .IP 12212 .nf 12213 \f[C] 12214 rclone rc vfs/forget file=path/to/file dir=path/to/dir 12215 \f[R] 12216 .fi 12217 .SS VFS File Buffering 12218 .PP 12219 The \f[C]--buffer-size\f[R] flag determines the amount of memory, that 12220 will be used to buffer data in advance. 12221 .PP 12222 Each open file will try to keep the specified amount of data in memory 12223 at all times. 12224 The buffered data is bound to one open file and won\[aq]t be shared. 12225 .PP 12226 This flag is a upper limit for the used memory per open file. 12227 The buffer will only use memory for data that is downloaded but not not 12228 yet read. 12229 If the buffer is empty, only a small amount of memory will be used. 12230 .PP 12231 The maximum memory used by rclone for buffering can be up to 12232 \f[C]--buffer-size * open files\f[R]. 12233 .SS VFS File Caching 12234 .PP 12235 These flags control the VFS file caching options. 12236 File caching is necessary to make the VFS layer appear compatible with a 12237 normal file system. 12238 It can be disabled at the cost of some compatibility. 12239 .PP 12240 For example you\[aq]ll need to enable VFS caching if you want to read 12241 and write simultaneously to a file. 12242 See below for more details. 12243 .PP 12244 Note that the VFS cache is separate from the cache backend and you may 12245 find that you need one or the other or both. 12246 .IP 12247 .nf 12248 \f[C] 12249 --cache-dir string Directory rclone will use for caching. 12250 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off) 12251 --vfs-cache-max-age duration Max time since last access of objects in the cache (default 1h0m0s) 12252 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache (default off) 12253 --vfs-cache-min-free-space SizeSuffix Target minimum free space on the disk containing the cache (default off) 12254 --vfs-cache-poll-interval duration Interval to poll the cache for stale objects (default 1m0s) 12255 --vfs-write-back duration Time to writeback files after last use when using cache (default 5s) 12256 \f[R] 12257 .fi 12258 .PP 12259 If run with \f[C]-vv\f[R] rclone will print the location of the file 12260 cache. 12261 The files are stored in the user cache file area which is OS dependent 12262 but can be controlled with \f[C]--cache-dir\f[R] or setting the 12263 appropriate environment variable. 12264 .PP 12265 The cache has 4 different modes selected by \f[C]--vfs-cache-mode\f[R]. 12266 The higher the cache mode the more compatible rclone becomes at the cost 12267 of using disk space. 12268 .PP 12269 Note that files are written back to the remote only when they are closed 12270 and if they haven\[aq]t been accessed for \f[C]--vfs-write-back\f[R] 12271 seconds. 12272 If rclone is quit or dies with files that haven\[aq]t been uploaded, 12273 these will be uploaded next time rclone is run with the same flags. 12274 .PP 12275 If using \f[C]--vfs-cache-max-size\f[R] or 12276 \f[C]--vfs-cache-min-free-size\f[R] note that the cache may exceed these 12277 quotas for two reasons. 12278 Firstly because it is only checked every 12279 \f[C]--vfs-cache-poll-interval\f[R]. 12280 Secondly because open files cannot be evicted from the cache. 12281 When \f[C]--vfs-cache-max-size\f[R] or 12282 \f[C]--vfs-cache-min-free-size\f[R] is exceeded, rclone will attempt to 12283 evict the least accessed files from the cache first. 12284 rclone will start with files that haven\[aq]t been accessed for the 12285 longest. 12286 This cache flushing strategy is efficient and more relevant files are 12287 likely to remain cached. 12288 .PP 12289 The \f[C]--vfs-cache-max-age\f[R] will evict files from the cache after 12290 the set time since last access has passed. 12291 The default value of 1 hour will start evicting files from cache that 12292 haven\[aq]t been accessed for 1 hour. 12293 When a cached file is accessed the 1 hour timer is reset to 0 and will 12294 wait for 1 more hour before evicting. 12295 Specify the time with standard notation, s, m, h, d, w . 12296 .PP 12297 You \f[B]should not\f[R] run two copies of rclone using the same VFS 12298 cache with the same or overlapping remotes if using 12299 \f[C]--vfs-cache-mode > off\f[R]. 12300 This can potentially cause data corruption if you do. 12301 You can work around this by giving each rclone its own cache hierarchy 12302 with \f[C]--cache-dir\f[R]. 12303 You don\[aq]t need to worry about this if the remotes in use don\[aq]t 12304 overlap. 12305 .SS --vfs-cache-mode off 12306 .PP 12307 In this mode (the default) the cache will read directly from the remote 12308 and write directly to the remote without caching anything on disk. 12309 .PP 12310 This will mean some operations are not possible 12311 .IP \[bu] 2 12312 Files can\[aq]t be opened for both read AND write 12313 .IP \[bu] 2 12314 Files opened for write can\[aq]t be seeked 12315 .IP \[bu] 2 12316 Existing files opened for write must have O_TRUNC set 12317 .IP \[bu] 2 12318 Files open for read with O_TRUNC will be opened write only 12319 .IP \[bu] 2 12320 Files open for write only will behave as if O_TRUNC was supplied 12321 .IP \[bu] 2 12322 Open modes O_APPEND, O_TRUNC are ignored 12323 .IP \[bu] 2 12324 If an upload fails it can\[aq]t be retried 12325 .SS --vfs-cache-mode minimal 12326 .PP 12327 This is very similar to \[dq]off\[dq] except that files opened for read 12328 AND write will be buffered to disk. 12329 This means that files opened for write will be a lot more compatible, 12330 but uses the minimal disk space. 12331 .PP 12332 These operations are not possible 12333 .IP \[bu] 2 12334 Files opened for write only can\[aq]t be seeked 12335 .IP \[bu] 2 12336 Existing files opened for write must have O_TRUNC set 12337 .IP \[bu] 2 12338 Files opened for write only will ignore O_APPEND, O_TRUNC 12339 .IP \[bu] 2 12340 If an upload fails it can\[aq]t be retried 12341 .SS --vfs-cache-mode writes 12342 .PP 12343 In this mode files opened for read only are still read directly from the 12344 remote, write only and read/write files are buffered to disk first. 12345 .PP 12346 This mode should support all normal file system operations. 12347 .PP 12348 If an upload fails it will be retried at exponentially increasing 12349 intervals up to 1 minute. 12350 .SS --vfs-cache-mode full 12351 .PP 12352 In this mode all reads and writes are buffered to and from disk. 12353 When data is read from the remote this is buffered to disk as well. 12354 .PP 12355 In this mode the files in the cache will be sparse files and rclone will 12356 keep track of which bits of the files it has downloaded. 12357 .PP 12358 So if an application only reads the starts of each file, then rclone 12359 will only buffer the start of the file. 12360 These files will appear to be their full size in the cache, but they 12361 will be sparse files with only the data that has been downloaded present 12362 in them. 12363 .PP 12364 This mode should support all normal file system operations and is 12365 otherwise identical to \f[C]--vfs-cache-mode\f[R] writes. 12366 .PP 12367 When reading a file rclone will read \f[C]--buffer-size\f[R] plus 12368 \f[C]--vfs-read-ahead\f[R] bytes ahead. 12369 The \f[C]--buffer-size\f[R] is buffered in memory whereas the 12370 \f[C]--vfs-read-ahead\f[R] is buffered on disk. 12371 .PP 12372 When using this mode it is recommended that \f[C]--buffer-size\f[R] is 12373 not set too large and \f[C]--vfs-read-ahead\f[R] is set large if 12374 required. 12375 .PP 12376 \f[B]IMPORTANT\f[R] not all file systems support sparse files. 12377 In particular FAT/exFAT do not. 12378 Rclone will perform very badly if the cache directory is on a filesystem 12379 which doesn\[aq]t support sparse files and it will log an ERROR message 12380 if one is detected. 12381 .SS Fingerprinting 12382 .PP 12383 Various parts of the VFS use fingerprinting to see if a local file copy 12384 has changed relative to a remote file. 12385 Fingerprints are made from: 12386 .IP \[bu] 2 12387 size 12388 .IP \[bu] 2 12389 modification time 12390 .IP \[bu] 2 12391 hash 12392 .PP 12393 where available on an object. 12394 .PP 12395 On some backends some of these attributes are slow to read (they take an 12396 extra API call per object, or extra work per object). 12397 .PP 12398 For example \f[C]hash\f[R] is slow with the \f[C]local\f[R] and 12399 \f[C]sftp\f[R] backends as they have to read the entire file and hash 12400 it, and \f[C]modtime\f[R] is slow with the \f[C]s3\f[R], 12401 \f[C]swift\f[R], \f[C]ftp\f[R] and \f[C]qinqstor\f[R] backends because 12402 they need to do an extra API call to fetch it. 12403 .PP 12404 If you use the \f[C]--vfs-fast-fingerprint\f[R] flag then rclone will 12405 not include the slow operations in the fingerprint. 12406 This makes the fingerprinting less accurate but much faster and will 12407 improve the opening time of cached files. 12408 .PP 12409 If you are running a vfs cache over \f[C]local\f[R], \f[C]s3\f[R] or 12410 \f[C]swift\f[R] backends then using this flag is recommended. 12411 .PP 12412 Note that if you change the value of this flag, the fingerprints of the 12413 files in the cache may be invalidated and the files will need to be 12414 downloaded again. 12415 .SS VFS Chunked Reading 12416 .PP 12417 When rclone reads files from a remote it reads them in chunks. 12418 This means that rather than requesting the whole file rclone reads the 12419 chunk specified. 12420 This can reduce the used download quota for some remotes by requesting 12421 only chunks from the remote that are actually read, at the cost of an 12422 increased number of requests. 12423 .PP 12424 These flags control the chunking: 12425 .IP 12426 .nf 12427 \f[C] 12428 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks (default 128M) 12429 --vfs-read-chunk-size-limit SizeSuffix Max chunk doubling size (default off) 12430 \f[R] 12431 .fi 12432 .PP 12433 Rclone will start reading a chunk of size 12434 \f[C]--vfs-read-chunk-size\f[R], and then double the size for each read. 12435 When \f[C]--vfs-read-chunk-size-limit\f[R] is specified, and greater 12436 than \f[C]--vfs-read-chunk-size\f[R], the chunk size for each open file 12437 will get doubled only until the specified value is reached. 12438 If the value is \[dq]off\[dq], which is the default, the limit is 12439 disabled and the chunk size will grow indefinitely. 12440 .PP 12441 With \f[C]--vfs-read-chunk-size 100M\f[R] and 12442 \f[C]--vfs-read-chunk-size-limit 0\f[R] the following parts will be 12443 downloaded: 0-100M, 100M-200M, 200M-300M, 300M-400M and so on. 12444 When \f[C]--vfs-read-chunk-size-limit 500M\f[R] is specified, the result 12445 would be 0-100M, 100M-300M, 300M-700M, 700M-1200M, 1200M-1700M and so 12446 on. 12447 .PP 12448 Setting \f[C]--vfs-read-chunk-size\f[R] to \f[C]0\f[R] or \[dq]off\[dq] 12449 disables chunked reading. 12450 .SS VFS Performance 12451 .PP 12452 These flags may be used to enable/disable features of the VFS for 12453 performance or other reasons. 12454 See also the chunked reading feature. 12455 .PP 12456 In particular S3 and Swift benefit hugely from the 12457 \f[C]--no-modtime\f[R] flag (or use \f[C]--use-server-modtime\f[R] for a 12458 slightly different effect) as each read of the modification time takes a 12459 transaction. 12460 .IP 12461 .nf 12462 \f[C] 12463 --no-checksum Don\[aq]t compare checksums on up/download. 12464 --no-modtime Don\[aq]t read/write the modification time (can speed things up). 12465 --no-seek Don\[aq]t allow seeking in files. 12466 --read-only Only allow read-only access. 12467 \f[R] 12468 .fi 12469 .PP 12470 Sometimes rclone is delivered reads or writes out of order. 12471 Rather than seeking rclone will wait a short time for the in sequence 12472 read or write to come in. 12473 These flags only come into effect when not using an on disk cache file. 12474 .IP 12475 .nf 12476 \f[C] 12477 --vfs-read-wait duration Time to wait for in-sequence read before seeking (default 20ms) 12478 --vfs-write-wait duration Time to wait for in-sequence write before giving error (default 1s) 12479 \f[R] 12480 .fi 12481 .PP 12482 When using VFS write caching (\f[C]--vfs-cache-mode\f[R] with value 12483 writes or full), the global flag \f[C]--transfers\f[R] can be set to 12484 adjust the number of parallel uploads of modified files from the cache 12485 (the related global flag \f[C]--checkers\f[R] has no effect on the VFS). 12486 .IP 12487 .nf 12488 \f[C] 12489 --transfers int Number of file transfers to run in parallel (default 4) 12490 \f[R] 12491 .fi 12492 .SS VFS Case Sensitivity 12493 .PP 12494 Linux file systems are case-sensitive: two files can differ only by 12495 case, and the exact case must be used when opening a file. 12496 .PP 12497 File systems in modern Windows are case-insensitive but case-preserving: 12498 although existing files can be opened using any case, the exact case 12499 used to create the file is preserved and available for programs to 12500 query. 12501 It is not allowed for two files in the same directory to differ only by 12502 case. 12503 .PP 12504 Usually file systems on macOS are case-insensitive. 12505 It is possible to make macOS file systems case-sensitive but that is not 12506 the default. 12507 .PP 12508 The \f[C]--vfs-case-insensitive\f[R] VFS flag controls how rclone 12509 handles these two cases. 12510 If its value is \[dq]false\[dq], rclone passes file names to the remote 12511 as-is. 12512 If the flag is \[dq]true\[dq] (or appears without a value on the command 12513 line), rclone may perform a \[dq]fixup\[dq] as explained below. 12514 .PP 12515 The user may specify a file name to open/delete/rename/etc with a case 12516 different than what is stored on the remote. 12517 If an argument refers to an existing file with exactly the same name, 12518 then the case of the existing file on the disk will be used. 12519 However, if a file name with exactly the same name is not found but a 12520 name differing only by case exists, rclone will transparently fixup the 12521 name. 12522 This fixup happens only when an existing file is requested. 12523 Case sensitivity of file names created anew by rclone is controlled by 12524 the underlying remote. 12525 .PP 12526 Note that case sensitivity of the operating system running rclone (the 12527 target) may differ from case sensitivity of a file system presented by 12528 rclone (the source). 12529 The flag controls whether \[dq]fixup\[dq] is performed to satisfy the 12530 target. 12531 .PP 12532 If the flag is not provided on the command line, then its default value 12533 depends on the operating system where rclone runs: \[dq]true\[dq] on 12534 Windows and macOS, \[dq]false\[dq] otherwise. 12535 If the flag is provided without a value, then it is \[dq]true\[dq]. 12536 .PP 12537 The \f[C]--no-unicode-normalization\f[R] flag controls whether a similar 12538 \[dq]fixup\[dq] is performed for filenames that differ but are 12539 canonically 12540 equivalent (https://en.wikipedia.org/wiki/Unicode_equivalence) with 12541 respect to unicode. 12542 Unicode normalization can be particularly helpful for users of macOS, 12543 which prefers form NFD instead of the NFC used by most other platforms. 12544 It is therefore highly recommended to keep the default of 12545 \f[C]false\f[R] on macOS, to avoid encoding compatibility issues. 12546 .PP 12547 In the (probably unlikely) event that a directory has multiple duplicate 12548 filenames after applying case and unicode normalization, the 12549 \f[C]--vfs-block-norm-dupes\f[R] flag allows hiding these duplicates. 12550 This comes with a performance tradeoff, as rclone will have to scan the 12551 entire directory for duplicates when listing a directory. 12552 For this reason, it is recommended to leave this disabled if not needed. 12553 However, macOS users may wish to consider using it, as otherwise, if a 12554 remote directory contains both NFC and NFD versions of the same 12555 filename, an odd situation will occur: both versions of the file will be 12556 visible in the mount, and both will appear to be editable, however, 12557 editing either version will actually result in only the NFD version 12558 getting edited under the hood. 12559 \f[C]--vfs-block- norm-dupes\f[R] prevents this confusion by detecting 12560 this scenario, hiding the duplicates, and logging an error, similar to 12561 how this is handled in \f[C]rclone sync\f[R]. 12562 .SS VFS Disk Options 12563 .PP 12564 This flag allows you to manually set the statistics about the filing 12565 system. 12566 It can be useful when those statistics cannot be read correctly 12567 automatically. 12568 .IP 12569 .nf 12570 \f[C] 12571 --vfs-disk-space-total-size Manually set the total disk space size (example: 256G, default: -1) 12572 \f[R] 12573 .fi 12574 .SS Alternate report of used bytes 12575 .PP 12576 Some backends, most notably S3, do not report the amount of bytes used. 12577 If you need this information to be available when running \f[C]df\f[R] 12578 on the filesystem, then pass the flag \f[C]--vfs-used-is-size\f[R] to 12579 rclone. 12580 With this flag set, instead of relying on the backend to report this 12581 information, rclone will scan the whole remote similar to 12582 \f[C]rclone size\f[R] and compute the total used space itself. 12583 .PP 12584 \f[I]WARNING.\f[R] Contrary to \f[C]rclone size\f[R], this flag ignores 12585 filters so that the result is accurate. 12586 However, this is very inefficient and may cost lots of API calls 12587 resulting in extra charges. 12588 Use it as a last resort and only with caching. 12589 .IP 12590 .nf 12591 \f[C] 12592 rclone serve nfs remote:path [flags] 12593 \f[R] 12594 .fi 12595 .SS Options 12596 .IP 12597 .nf 12598 \f[C] 12599 --addr string IPaddress:Port or :Port to bind server to 12600 --dir-cache-time Duration Time to cache directory entries for (default 5m0s) 12601 --dir-perms FileMode Directory permissions (default 0777) 12602 --file-perms FileMode File permissions (default 0666) 12603 --gid uint32 Override the gid field set by the filesystem (not supported on Windows) (default 1000) 12604 -h, --help help for nfs 12605 --nfs-cache-handle-limit int max file handles cached simultaneously (min 5) (default 1000000) 12606 --no-checksum Don\[aq]t compare checksums on up/download 12607 --no-modtime Don\[aq]t read/write the modification time (can speed things up) 12608 --no-seek Don\[aq]t allow seeking in files 12609 --poll-interval Duration Time to wait between polling for changes, must be smaller than dir-cache-time and only on supported remotes (set 0 to disable) (default 1m0s) 12610 --read-only Only allow read-only access 12611 --uid uint32 Override the uid field set by the filesystem (not supported on Windows) (default 1000) 12612 --umask int Override the permission bits set by the filesystem (not supported on Windows) (default 2) 12613 --vfs-block-norm-dupes If duplicate filenames exist in the same directory (after normalization), log an error and hide the duplicates (may have a performance cost) 12614 --vfs-cache-max-age Duration Max time since last access of objects in the cache (default 1h0m0s) 12615 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache (default off) 12616 --vfs-cache-min-free-space SizeSuffix Target minimum free space on the disk containing the cache (default off) 12617 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off) 12618 --vfs-cache-poll-interval Duration Interval to poll the cache for stale objects (default 1m0s) 12619 --vfs-case-insensitive If a file name not found, find a case insensitive match 12620 --vfs-disk-space-total-size SizeSuffix Specify the total space of disk (default off) 12621 --vfs-fast-fingerprint Use fast (less accurate) fingerprints for change detection 12622 --vfs-read-ahead SizeSuffix Extra read ahead over --buffer-size when using cache-mode full 12623 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks (default 128Mi) 12624 --vfs-read-chunk-size-limit SizeSuffix If greater than --vfs-read-chunk-size, double the chunk size after each chunk read, until the limit is reached (\[aq]off\[aq] is unlimited) (default off) 12625 --vfs-read-wait Duration Time to wait for in-sequence read before seeking (default 20ms) 12626 --vfs-refresh Refreshes the directory cache recursively in the background on start 12627 --vfs-used-is-size rclone size Use the rclone size algorithm for Used size 12628 --vfs-write-back Duration Time to writeback files after last use when using cache (default 5s) 12629 --vfs-write-wait Duration Time to wait for in-sequence write before giving error (default 1s) 12630 \f[R] 12631 .fi 12632 .SS Filter Options 12633 .PP 12634 Flags for filtering directory listings. 12635 .IP 12636 .nf 12637 \f[C] 12638 --delete-excluded Delete files on dest excluded from sync 12639 --exclude stringArray Exclude files matching pattern 12640 --exclude-from stringArray Read file exclude patterns from file (use - to read from stdin) 12641 --exclude-if-present stringArray Exclude directories if filename is present 12642 --files-from stringArray Read list of source-file names from file (use - to read from stdin) 12643 --files-from-raw stringArray Read list of source-file names from file without any processing of lines (use - to read from stdin) 12644 -f, --filter stringArray Add a file filtering rule 12645 --filter-from stringArray Read file filtering patterns from a file (use - to read from stdin) 12646 --ignore-case Ignore case in filters (case insensitive) 12647 --include stringArray Include files matching pattern 12648 --include-from stringArray Read file include patterns from file (use - to read from stdin) 12649 --max-age Duration Only transfer files younger than this in s or suffix ms|s|m|h|d|w|M|y (default off) 12650 --max-depth int If set limits the recursion depth to this (default -1) 12651 --max-size SizeSuffix Only transfer files smaller than this in KiB or suffix B|K|M|G|T|P (default off) 12652 --metadata-exclude stringArray Exclude metadatas matching pattern 12653 --metadata-exclude-from stringArray Read metadata exclude patterns from file (use - to read from stdin) 12654 --metadata-filter stringArray Add a metadata filtering rule 12655 --metadata-filter-from stringArray Read metadata filtering patterns from a file (use - to read from stdin) 12656 --metadata-include stringArray Include metadatas matching pattern 12657 --metadata-include-from stringArray Read metadata include patterns from file (use - to read from stdin) 12658 --min-age Duration Only transfer files older than this in s or suffix ms|s|m|h|d|w|M|y (default off) 12659 --min-size SizeSuffix Only transfer files bigger than this in KiB or suffix B|K|M|G|T|P (default off) 12660 \f[R] 12661 .fi 12662 .PP 12663 See the global flags page (https://rclone.org/flags/) for global options 12664 not listed here. 12665 .SH SEE ALSO 12666 .IP \[bu] 2 12667 rclone serve (https://rclone.org/commands/rclone_serve/) - Serve a 12668 remote over a protocol. 12669 .SH rclone serve restic 12670 .PP 12671 Serve the remote for restic\[aq]s REST API. 12672 .SS Synopsis 12673 .PP 12674 Run a basic web server to serve a remote over restic\[aq]s REST backend 12675 API over HTTP. 12676 This allows restic to use rclone as a data storage mechanism for cloud 12677 providers that restic does not support directly. 12678 .PP 12679 Restic (https://restic.net/) is a command-line program for doing 12680 backups. 12681 .PP 12682 The server will log errors. 12683 Use -v to see access logs. 12684 .PP 12685 \f[C]--bwlimit\f[R] will be respected for file transfers. 12686 Use \f[C]--stats\f[R] to control the stats printing. 12687 .SS Setting up rclone for use by restic 12688 .PP 12689 First set up a remote for your chosen cloud 12690 provider (https://rclone.org/docs/#configure). 12691 .PP 12692 Once you have set up the remote, check it is working with, for example 12693 \[dq]rclone lsd remote:\[dq]. 12694 You may have called the remote something other than \[dq]remote:\[dq] - 12695 just substitute whatever you called it in the following instructions. 12696 .PP 12697 Now start the rclone restic server 12698 .IP 12699 .nf 12700 \f[C] 12701 rclone serve restic -v remote:backup 12702 \f[R] 12703 .fi 12704 .PP 12705 Where you can replace \[dq]backup\[dq] in the above by whatever path in 12706 the remote you wish to use. 12707 .PP 12708 By default this will serve on \[dq]localhost:8080\[dq] you can change 12709 this with use of the \f[C]--addr\f[R] flag. 12710 .PP 12711 You might wish to start this server on boot. 12712 .PP 12713 Adding \f[C]--cache-objects=false\f[R] will cause rclone to stop caching 12714 objects returned from the List call. 12715 Caching is normally desirable as it speeds up downloading objects, saves 12716 transactions and uses very little memory. 12717 .SS Setting up restic to use rclone 12718 .PP 12719 Now you can follow the restic 12720 instructions (http://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html#rest-server) 12721 on setting up restic. 12722 .PP 12723 Note that you will need restic 0.8.2 or later to interoperate with 12724 rclone. 12725 .PP 12726 For the example above you will want to use 12727 \[dq]http://localhost:8080/\[dq] as the URL for the REST server. 12728 .PP 12729 For example: 12730 .IP 12731 .nf 12732 \f[C] 12733 $ export RESTIC_REPOSITORY=rest:http://localhost:8080/ 12734 $ export RESTIC_PASSWORD=yourpassword 12735 $ restic init 12736 created restic backend 8b1a4b56ae at rest:http://localhost:8080/ 12737 12738 Please note that knowledge of your password is required to access 12739 the repository. Losing your password means that your data is 12740 irrecoverably lost. 12741 $ restic backup /path/to/files/to/backup 12742 scan [/path/to/files/to/backup] 12743 scanned 189 directories, 312 files in 0:00 12744 [0:00] 100.00% 38.128 MiB / 38.128 MiB 501 / 501 items 0 errors ETA 0:00 12745 duration: 0:00 12746 snapshot 45c8fdd8 saved 12747 \f[R] 12748 .fi 12749 .SS Multiple repositories 12750 .PP 12751 Note that you can use the endpoint to host multiple repositories. 12752 Do this by adding a directory name or path after the URL. 12753 Note that these \f[B]must\f[R] end with /. 12754 Eg 12755 .IP 12756 .nf 12757 \f[C] 12758 $ export RESTIC_REPOSITORY=rest:http://localhost:8080/user1repo/ 12759 # backup user1 stuff 12760 $ export RESTIC_REPOSITORY=rest:http://localhost:8080/user2repo/ 12761 # backup user2 stuff 12762 \f[R] 12763 .fi 12764 .SS Private repositories 12765 .PP 12766 The\f[C]--private-repos\f[R] flag can be used to limit users to 12767 repositories starting with a path of \f[C]/<username>/\f[R]. 12768 .SS Server options 12769 .PP 12770 Use \f[C]--addr\f[R] to specify which IP address and port the server 12771 should listen on, eg \f[C]--addr 1.2.3.4:8000\f[R] or 12772 \f[C]--addr :8080\f[R] to listen to all IPs. 12773 By default it only listens on localhost. 12774 You can use port :0 to let the OS choose an available port. 12775 .PP 12776 If you set \f[C]--addr\f[R] to listen on a public or LAN accessible IP 12777 address then using Authentication is advised - see the next section for 12778 info. 12779 .PP 12780 You can use a unix socket by setting the url to 12781 \f[C]unix:///path/to/socket\f[R] or just by using an absolute path name. 12782 Note that unix sockets bypass the authentication - this is expected to 12783 be done with file system permissions. 12784 .PP 12785 \f[C]--addr\f[R] may be repeated to listen on multiple 12786 IPs/ports/sockets. 12787 .PP 12788 \f[C]--server-read-timeout\f[R] and \f[C]--server-write-timeout\f[R] can 12789 be used to control the timeouts on the server. 12790 Note that this is the total time for a transfer. 12791 .PP 12792 \f[C]--max-header-bytes\f[R] controls the maximum number of bytes the 12793 server will accept in the HTTP header. 12794 .PP 12795 \f[C]--baseurl\f[R] controls the URL prefix that rclone serves from. 12796 By default rclone will serve from the root. 12797 If you used \f[C]--baseurl \[dq]/rclone\[dq]\f[R] then rclone would 12798 serve from a URL starting with \[dq]/rclone/\[dq]. 12799 This is useful if you wish to proxy rclone serve. 12800 Rclone automatically inserts leading and trailing \[dq]/\[dq] on 12801 \f[C]--baseurl\f[R], so \f[C]--baseurl \[dq]rclone\[dq]\f[R], 12802 \f[C]--baseurl \[dq]/rclone\[dq]\f[R] and 12803 \f[C]--baseurl \[dq]/rclone/\[dq]\f[R] are all treated identically. 12804 .SS TLS (SSL) 12805 .PP 12806 By default this will serve over http. 12807 If you want you can serve over https. 12808 You will need to supply the \f[C]--cert\f[R] and \f[C]--key\f[R] flags. 12809 If you wish to do client side certificate validation then you will need 12810 to supply \f[C]--client-ca\f[R] also. 12811 .PP 12812 \f[C]--cert\f[R] should be a either a PEM encoded certificate or a 12813 concatenation of that with the CA certificate. 12814 \f[C]--key\f[R] should be the PEM encoded private key and 12815 \f[C]--client-ca\f[R] should be the PEM encoded client certificate 12816 authority certificate. 12817 .PP 12818 --min-tls-version is minimum TLS version that is acceptable. 12819 Valid values are \[dq]tls1.0\[dq], \[dq]tls1.1\[dq], \[dq]tls1.2\[dq] 12820 and \[dq]tls1.3\[dq] (default \[dq]tls1.0\[dq]). 12821 .SS Authentication 12822 .PP 12823 By default this will serve files without needing a login. 12824 .PP 12825 You can either use an htpasswd file which can take lots of users, or set 12826 a single username and password with the \f[C]--user\f[R] and 12827 \f[C]--pass\f[R] flags. 12828 .PP 12829 If no static users are configured by either of the above methods, and 12830 client certificates are required by the \f[C]--client-ca\f[R] flag 12831 passed to the server, the client certificate common name will be 12832 considered as the username. 12833 .PP 12834 Use \f[C]--htpasswd /path/to/htpasswd\f[R] to provide an htpasswd file. 12835 This is in standard apache format and supports MD5, SHA1 and BCrypt for 12836 basic authentication. 12837 Bcrypt is recommended. 12838 .PP 12839 To create an htpasswd file: 12840 .IP 12841 .nf 12842 \f[C] 12843 touch htpasswd 12844 htpasswd -B htpasswd user 12845 htpasswd -B htpasswd anotherUser 12846 \f[R] 12847 .fi 12848 .PP 12849 The password file can be updated while rclone is running. 12850 .PP 12851 Use \f[C]--realm\f[R] to set the authentication realm. 12852 .PP 12853 Use \f[C]--salt\f[R] to change the password hashing salt from the 12854 default. 12855 .IP 12856 .nf 12857 \f[C] 12858 rclone serve restic remote:path [flags] 12859 \f[R] 12860 .fi 12861 .SS Options 12862 .IP 12863 .nf 12864 \f[C] 12865 --addr stringArray IPaddress:Port or :Port to bind server to (default [127.0.0.1:8080]) 12866 --allow-origin string Origin which cross-domain request (CORS) can be executed from 12867 --append-only Disallow deletion of repository data 12868 --baseurl string Prefix for URLs - leave blank for root 12869 --cache-objects Cache listed objects (default true) 12870 --cert string TLS PEM key (concatenation of certificate and CA certificate) 12871 --client-ca string Client certificate authority to verify clients with 12872 -h, --help help for restic 12873 --htpasswd string A htpasswd file - if not provided no authentication is done 12874 --key string TLS PEM Private key 12875 --max-header-bytes int Maximum size of request header (default 4096) 12876 --min-tls-version string Minimum TLS version that is acceptable (default \[dq]tls1.0\[dq]) 12877 --pass string Password for authentication 12878 --private-repos Users can only access their private repo 12879 --realm string Realm for authentication 12880 --salt string Password hashing salt (default \[dq]dlPL2MqE\[dq]) 12881 --server-read-timeout Duration Timeout for server reading data (default 1h0m0s) 12882 --server-write-timeout Duration Timeout for server writing data (default 1h0m0s) 12883 --stdio Run an HTTP2 server on stdin/stdout 12884 --user string User name for authentication 12885 \f[R] 12886 .fi 12887 .PP 12888 See the global flags page (https://rclone.org/flags/) for global options 12889 not listed here. 12890 .SH SEE ALSO 12891 .IP \[bu] 2 12892 rclone serve (https://rclone.org/commands/rclone_serve/) - Serve a 12893 remote over a protocol. 12894 .SH rclone serve s3 12895 .PP 12896 Serve remote:path over s3. 12897 .SS Synopsis 12898 .PP 12899 \f[C]serve s3\f[R] implements a basic s3 server that serves a remote via 12900 s3. 12901 This can be viewed with an s3 client, or you can make an s3 type 12902 remote (https://rclone.org/s3/) to read and write to it with rclone. 12903 .PP 12904 \f[C]serve s3\f[R] is considered \f[B]Experimental\f[R] so use with 12905 care. 12906 .PP 12907 S3 server supports Signature Version 4 authentication. 12908 Just use \f[C]--auth-key accessKey,secretKey\f[R] and set the 12909 \f[C]Authorization\f[R] header correctly in the request. 12910 (See the AWS 12911 docs (https://docs.aws.amazon.com/general/latest/gr/signature-version-4.html)). 12912 .PP 12913 \f[C]--auth-key\f[R] can be repeated for multiple auth pairs. 12914 If \f[C]--auth-key\f[R] is not provided then \f[C]serve s3\f[R] will 12915 allow anonymous access. 12916 .PP 12917 Please note that some clients may require HTTPS endpoints. 12918 See the SSL docs for more information. 12919 .PP 12920 This command uses the VFS directory cache. 12921 All the functionality will work with \f[C]--vfs-cache-mode off\f[R]. 12922 Using \f[C]--vfs-cache-mode full\f[R] (or \f[C]writes\f[R]) can be used 12923 to cache objects locally to improve performance. 12924 .PP 12925 Use \f[C]--force-path-style=false\f[R] if you want to use the bucket 12926 name as a part of the hostname (such as mybucket.local) 12927 .PP 12928 Use \f[C]--etag-hash\f[R] if you want to change the hash uses for the 12929 \f[C]ETag\f[R]. 12930 Note that using anything other than \f[C]MD5\f[R] (the default) is 12931 likely to cause problems for S3 clients which rely on the Etag being the 12932 MD5. 12933 .SS Quickstart 12934 .PP 12935 For a simple set up, to serve \f[C]remote:path\f[R] over s3, run the 12936 server like this: 12937 .IP 12938 .nf 12939 \f[C] 12940 rclone serve s3 --auth-key ACCESS_KEY_ID,SECRET_ACCESS_KEY remote:path 12941 \f[R] 12942 .fi 12943 .PP 12944 This will be compatible with an rclone remote which is defined like 12945 this: 12946 .IP 12947 .nf 12948 \f[C] 12949 [serves3] 12950 type = s3 12951 provider = Rclone 12952 endpoint = http://127.0.0.1:8080/ 12953 access_key_id = ACCESS_KEY_ID 12954 secret_access_key = SECRET_ACCESS_KEY 12955 use_multipart_uploads = false 12956 \f[R] 12957 .fi 12958 .PP 12959 Note that setting \f[C]disable_multipart_uploads = true\f[R] is to work 12960 around a bug which will be fixed in due course. 12961 .SS Bugs 12962 .PP 12963 When uploading multipart files \f[C]serve s3\f[R] holds all the parts in 12964 memory (see #7453 (https://github.com/artpar/artpar/issues/7453)). 12965 This is a limitaton of the library rclone uses for serving S3 and will 12966 hopefully be fixed at some point. 12967 .PP 12968 Multipart server side copies do not work (see 12969 #7454 (https://github.com/artpar/artpar/issues/7454)). 12970 These take a very long time and eventually fail. 12971 The default threshold for multipart server side copies is 5G which is 12972 the maximum it can be, so files above this side will fail to be server 12973 side copied. 12974 .PP 12975 For a current list of \f[C]serve s3\f[R] bugs see the serve 12976 s3 (https://github.com/artpar/artpar/labels/serve%20s3) bug category on 12977 GitHub. 12978 .SS Limitations 12979 .PP 12980 \f[C]serve s3\f[R] will treat all directories in the root as buckets and 12981 ignore all files in the root. 12982 You can use \f[C]CreateBucket\f[R] to create folders under the root, but 12983 you can\[aq]t create empty folders under other folders not in the root. 12984 .PP 12985 When using \f[C]PutObject\f[R] or \f[C]DeleteObject\f[R], rclone will 12986 automatically create or clean up empty folders. 12987 If you don\[aq]t want to clean up empty folders automatically, use 12988 \f[C]--no-cleanup\f[R]. 12989 .PP 12990 When using \f[C]ListObjects\f[R], rclone will use \f[C]/\f[R] when the 12991 delimiter is empty. 12992 This reduces backend requests with no effect on most operations, but if 12993 the delimiter is something other than \f[C]/\f[R] and empty, rclone will 12994 do a full recursive search of the backend, which can take some time. 12995 .PP 12996 Versioning is not currently supported. 12997 .PP 12998 Metadata will only be saved in memory other than the rclone 12999 \f[C]mtime\f[R] metadata which will be set as the modification time of 13000 the file. 13001 .SS Supported operations 13002 .PP 13003 \f[C]serve s3\f[R] currently supports the following operations. 13004 .IP \[bu] 2 13005 Bucket 13006 .RS 2 13007 .IP \[bu] 2 13008 \f[C]ListBuckets\f[R] 13009 .IP \[bu] 2 13010 \f[C]CreateBucket\f[R] 13011 .IP \[bu] 2 13012 \f[C]DeleteBucket\f[R] 13013 .RE 13014 .IP \[bu] 2 13015 Object 13016 .RS 2 13017 .IP \[bu] 2 13018 \f[C]HeadObject\f[R] 13019 .IP \[bu] 2 13020 \f[C]ListObjects\f[R] 13021 .IP \[bu] 2 13022 \f[C]GetObject\f[R] 13023 .IP \[bu] 2 13024 \f[C]PutObject\f[R] 13025 .IP \[bu] 2 13026 \f[C]DeleteObject\f[R] 13027 .IP \[bu] 2 13028 \f[C]DeleteObjects\f[R] 13029 .IP \[bu] 2 13030 \f[C]CreateMultipartUpload\f[R] 13031 .IP \[bu] 2 13032 \f[C]CompleteMultipartUpload\f[R] 13033 .IP \[bu] 2 13034 \f[C]AbortMultipartUpload\f[R] 13035 .IP \[bu] 2 13036 \f[C]CopyObject\f[R] 13037 .IP \[bu] 2 13038 \f[C]UploadPart\f[R] 13039 .RE 13040 .PP 13041 Other operations will return error \f[C]Unimplemented\f[R]. 13042 .SS Server options 13043 .PP 13044 Use \f[C]--addr\f[R] to specify which IP address and port the server 13045 should listen on, eg \f[C]--addr 1.2.3.4:8000\f[R] or 13046 \f[C]--addr :8080\f[R] to listen to all IPs. 13047 By default it only listens on localhost. 13048 You can use port :0 to let the OS choose an available port. 13049 .PP 13050 If you set \f[C]--addr\f[R] to listen on a public or LAN accessible IP 13051 address then using Authentication is advised - see the next section for 13052 info. 13053 .PP 13054 You can use a unix socket by setting the url to 13055 \f[C]unix:///path/to/socket\f[R] or just by using an absolute path name. 13056 Note that unix sockets bypass the authentication - this is expected to 13057 be done with file system permissions. 13058 .PP 13059 \f[C]--addr\f[R] may be repeated to listen on multiple 13060 IPs/ports/sockets. 13061 .PP 13062 \f[C]--server-read-timeout\f[R] and \f[C]--server-write-timeout\f[R] can 13063 be used to control the timeouts on the server. 13064 Note that this is the total time for a transfer. 13065 .PP 13066 \f[C]--max-header-bytes\f[R] controls the maximum number of bytes the 13067 server will accept in the HTTP header. 13068 .PP 13069 \f[C]--baseurl\f[R] controls the URL prefix that rclone serves from. 13070 By default rclone will serve from the root. 13071 If you used \f[C]--baseurl \[dq]/rclone\[dq]\f[R] then rclone would 13072 serve from a URL starting with \[dq]/rclone/\[dq]. 13073 This is useful if you wish to proxy rclone serve. 13074 Rclone automatically inserts leading and trailing \[dq]/\[dq] on 13075 \f[C]--baseurl\f[R], so \f[C]--baseurl \[dq]rclone\[dq]\f[R], 13076 \f[C]--baseurl \[dq]/rclone\[dq]\f[R] and 13077 \f[C]--baseurl \[dq]/rclone/\[dq]\f[R] are all treated identically. 13078 .SS TLS (SSL) 13079 .PP 13080 By default this will serve over http. 13081 If you want you can serve over https. 13082 You will need to supply the \f[C]--cert\f[R] and \f[C]--key\f[R] flags. 13083 If you wish to do client side certificate validation then you will need 13084 to supply \f[C]--client-ca\f[R] also. 13085 .PP 13086 \f[C]--cert\f[R] should be a either a PEM encoded certificate or a 13087 concatenation of that with the CA certificate. 13088 \f[C]--key\f[R] should be the PEM encoded private key and 13089 \f[C]--client-ca\f[R] should be the PEM encoded client certificate 13090 authority certificate. 13091 .PP 13092 --min-tls-version is minimum TLS version that is acceptable. 13093 Valid values are \[dq]tls1.0\[dq], \[dq]tls1.1\[dq], \[dq]tls1.2\[dq] 13094 and \[dq]tls1.3\[dq] (default \[dq]tls1.0\[dq]). 13095 ## VFS - Virtual File System 13096 .PP 13097 This command uses the VFS layer. 13098 This adapts the cloud storage objects that rclone uses into something 13099 which looks much more like a disk filing system. 13100 .PP 13101 Cloud storage objects have lots of properties which aren\[aq]t like disk 13102 files - you can\[aq]t extend them or write to the middle of them, so the 13103 VFS layer has to deal with that. 13104 Because there is no one right way of doing this there are various 13105 options explained below. 13106 .PP 13107 The VFS layer also implements a directory cache - this caches info about 13108 files and directories (but not the data) in memory. 13109 .SS VFS Directory Cache 13110 .PP 13111 Using the \f[C]--dir-cache-time\f[R] flag, you can control how long a 13112 directory should be considered up to date and not refreshed from the 13113 backend. 13114 Changes made through the VFS will appear immediately or invalidate the 13115 cache. 13116 .IP 13117 .nf 13118 \f[C] 13119 --dir-cache-time duration Time to cache directory entries for (default 5m0s) 13120 --poll-interval duration Time to wait between polling for changes. Must be smaller than dir-cache-time. Only on supported remotes. Set to 0 to disable (default 1m0s) 13121 \f[R] 13122 .fi 13123 .PP 13124 However, changes made directly on the cloud storage by the web interface 13125 or a different copy of rclone will only be picked up once the directory 13126 cache expires if the backend configured does not support polling for 13127 changes. 13128 If the backend supports polling, changes will be picked up within the 13129 polling interval. 13130 .PP 13131 You can send a \f[C]SIGHUP\f[R] signal to rclone for it to flush all 13132 directory caches, regardless of how old they are. 13133 Assuming only one rclone instance is running, you can reset the cache 13134 like this: 13135 .IP 13136 .nf 13137 \f[C] 13138 kill -SIGHUP $(pidof rclone) 13139 \f[R] 13140 .fi 13141 .PP 13142 If you configure rclone with a remote control then you can use rclone rc 13143 to flush the whole directory cache: 13144 .IP 13145 .nf 13146 \f[C] 13147 rclone rc vfs/forget 13148 \f[R] 13149 .fi 13150 .PP 13151 Or individual files or directories: 13152 .IP 13153 .nf 13154 \f[C] 13155 rclone rc vfs/forget file=path/to/file dir=path/to/dir 13156 \f[R] 13157 .fi 13158 .SS VFS File Buffering 13159 .PP 13160 The \f[C]--buffer-size\f[R] flag determines the amount of memory, that 13161 will be used to buffer data in advance. 13162 .PP 13163 Each open file will try to keep the specified amount of data in memory 13164 at all times. 13165 The buffered data is bound to one open file and won\[aq]t be shared. 13166 .PP 13167 This flag is a upper limit for the used memory per open file. 13168 The buffer will only use memory for data that is downloaded but not not 13169 yet read. 13170 If the buffer is empty, only a small amount of memory will be used. 13171 .PP 13172 The maximum memory used by rclone for buffering can be up to 13173 \f[C]--buffer-size * open files\f[R]. 13174 .SS VFS File Caching 13175 .PP 13176 These flags control the VFS file caching options. 13177 File caching is necessary to make the VFS layer appear compatible with a 13178 normal file system. 13179 It can be disabled at the cost of some compatibility. 13180 .PP 13181 For example you\[aq]ll need to enable VFS caching if you want to read 13182 and write simultaneously to a file. 13183 See below for more details. 13184 .PP 13185 Note that the VFS cache is separate from the cache backend and you may 13186 find that you need one or the other or both. 13187 .IP 13188 .nf 13189 \f[C] 13190 --cache-dir string Directory rclone will use for caching. 13191 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off) 13192 --vfs-cache-max-age duration Max time since last access of objects in the cache (default 1h0m0s) 13193 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache (default off) 13194 --vfs-cache-min-free-space SizeSuffix Target minimum free space on the disk containing the cache (default off) 13195 --vfs-cache-poll-interval duration Interval to poll the cache for stale objects (default 1m0s) 13196 --vfs-write-back duration Time to writeback files after last use when using cache (default 5s) 13197 \f[R] 13198 .fi 13199 .PP 13200 If run with \f[C]-vv\f[R] rclone will print the location of the file 13201 cache. 13202 The files are stored in the user cache file area which is OS dependent 13203 but can be controlled with \f[C]--cache-dir\f[R] or setting the 13204 appropriate environment variable. 13205 .PP 13206 The cache has 4 different modes selected by \f[C]--vfs-cache-mode\f[R]. 13207 The higher the cache mode the more compatible rclone becomes at the cost 13208 of using disk space. 13209 .PP 13210 Note that files are written back to the remote only when they are closed 13211 and if they haven\[aq]t been accessed for \f[C]--vfs-write-back\f[R] 13212 seconds. 13213 If rclone is quit or dies with files that haven\[aq]t been uploaded, 13214 these will be uploaded next time rclone is run with the same flags. 13215 .PP 13216 If using \f[C]--vfs-cache-max-size\f[R] or 13217 \f[C]--vfs-cache-min-free-size\f[R] note that the cache may exceed these 13218 quotas for two reasons. 13219 Firstly because it is only checked every 13220 \f[C]--vfs-cache-poll-interval\f[R]. 13221 Secondly because open files cannot be evicted from the cache. 13222 When \f[C]--vfs-cache-max-size\f[R] or 13223 \f[C]--vfs-cache-min-free-size\f[R] is exceeded, rclone will attempt to 13224 evict the least accessed files from the cache first. 13225 rclone will start with files that haven\[aq]t been accessed for the 13226 longest. 13227 This cache flushing strategy is efficient and more relevant files are 13228 likely to remain cached. 13229 .PP 13230 The \f[C]--vfs-cache-max-age\f[R] will evict files from the cache after 13231 the set time since last access has passed. 13232 The default value of 1 hour will start evicting files from cache that 13233 haven\[aq]t been accessed for 1 hour. 13234 When a cached file is accessed the 1 hour timer is reset to 0 and will 13235 wait for 1 more hour before evicting. 13236 Specify the time with standard notation, s, m, h, d, w . 13237 .PP 13238 You \f[B]should not\f[R] run two copies of rclone using the same VFS 13239 cache with the same or overlapping remotes if using 13240 \f[C]--vfs-cache-mode > off\f[R]. 13241 This can potentially cause data corruption if you do. 13242 You can work around this by giving each rclone its own cache hierarchy 13243 with \f[C]--cache-dir\f[R]. 13244 You don\[aq]t need to worry about this if the remotes in use don\[aq]t 13245 overlap. 13246 .SS --vfs-cache-mode off 13247 .PP 13248 In this mode (the default) the cache will read directly from the remote 13249 and write directly to the remote without caching anything on disk. 13250 .PP 13251 This will mean some operations are not possible 13252 .IP \[bu] 2 13253 Files can\[aq]t be opened for both read AND write 13254 .IP \[bu] 2 13255 Files opened for write can\[aq]t be seeked 13256 .IP \[bu] 2 13257 Existing files opened for write must have O_TRUNC set 13258 .IP \[bu] 2 13259 Files open for read with O_TRUNC will be opened write only 13260 .IP \[bu] 2 13261 Files open for write only will behave as if O_TRUNC was supplied 13262 .IP \[bu] 2 13263 Open modes O_APPEND, O_TRUNC are ignored 13264 .IP \[bu] 2 13265 If an upload fails it can\[aq]t be retried 13266 .SS --vfs-cache-mode minimal 13267 .PP 13268 This is very similar to \[dq]off\[dq] except that files opened for read 13269 AND write will be buffered to disk. 13270 This means that files opened for write will be a lot more compatible, 13271 but uses the minimal disk space. 13272 .PP 13273 These operations are not possible 13274 .IP \[bu] 2 13275 Files opened for write only can\[aq]t be seeked 13276 .IP \[bu] 2 13277 Existing files opened for write must have O_TRUNC set 13278 .IP \[bu] 2 13279 Files opened for write only will ignore O_APPEND, O_TRUNC 13280 .IP \[bu] 2 13281 If an upload fails it can\[aq]t be retried 13282 .SS --vfs-cache-mode writes 13283 .PP 13284 In this mode files opened for read only are still read directly from the 13285 remote, write only and read/write files are buffered to disk first. 13286 .PP 13287 This mode should support all normal file system operations. 13288 .PP 13289 If an upload fails it will be retried at exponentially increasing 13290 intervals up to 1 minute. 13291 .SS --vfs-cache-mode full 13292 .PP 13293 In this mode all reads and writes are buffered to and from disk. 13294 When data is read from the remote this is buffered to disk as well. 13295 .PP 13296 In this mode the files in the cache will be sparse files and rclone will 13297 keep track of which bits of the files it has downloaded. 13298 .PP 13299 So if an application only reads the starts of each file, then rclone 13300 will only buffer the start of the file. 13301 These files will appear to be their full size in the cache, but they 13302 will be sparse files with only the data that has been downloaded present 13303 in them. 13304 .PP 13305 This mode should support all normal file system operations and is 13306 otherwise identical to \f[C]--vfs-cache-mode\f[R] writes. 13307 .PP 13308 When reading a file rclone will read \f[C]--buffer-size\f[R] plus 13309 \f[C]--vfs-read-ahead\f[R] bytes ahead. 13310 The \f[C]--buffer-size\f[R] is buffered in memory whereas the 13311 \f[C]--vfs-read-ahead\f[R] is buffered on disk. 13312 .PP 13313 When using this mode it is recommended that \f[C]--buffer-size\f[R] is 13314 not set too large and \f[C]--vfs-read-ahead\f[R] is set large if 13315 required. 13316 .PP 13317 \f[B]IMPORTANT\f[R] not all file systems support sparse files. 13318 In particular FAT/exFAT do not. 13319 Rclone will perform very badly if the cache directory is on a filesystem 13320 which doesn\[aq]t support sparse files and it will log an ERROR message 13321 if one is detected. 13322 .SS Fingerprinting 13323 .PP 13324 Various parts of the VFS use fingerprinting to see if a local file copy 13325 has changed relative to a remote file. 13326 Fingerprints are made from: 13327 .IP \[bu] 2 13328 size 13329 .IP \[bu] 2 13330 modification time 13331 .IP \[bu] 2 13332 hash 13333 .PP 13334 where available on an object. 13335 .PP 13336 On some backends some of these attributes are slow to read (they take an 13337 extra API call per object, or extra work per object). 13338 .PP 13339 For example \f[C]hash\f[R] is slow with the \f[C]local\f[R] and 13340 \f[C]sftp\f[R] backends as they have to read the entire file and hash 13341 it, and \f[C]modtime\f[R] is slow with the \f[C]s3\f[R], 13342 \f[C]swift\f[R], \f[C]ftp\f[R] and \f[C]qinqstor\f[R] backends because 13343 they need to do an extra API call to fetch it. 13344 .PP 13345 If you use the \f[C]--vfs-fast-fingerprint\f[R] flag then rclone will 13346 not include the slow operations in the fingerprint. 13347 This makes the fingerprinting less accurate but much faster and will 13348 improve the opening time of cached files. 13349 .PP 13350 If you are running a vfs cache over \f[C]local\f[R], \f[C]s3\f[R] or 13351 \f[C]swift\f[R] backends then using this flag is recommended. 13352 .PP 13353 Note that if you change the value of this flag, the fingerprints of the 13354 files in the cache may be invalidated and the files will need to be 13355 downloaded again. 13356 .SS VFS Chunked Reading 13357 .PP 13358 When rclone reads files from a remote it reads them in chunks. 13359 This means that rather than requesting the whole file rclone reads the 13360 chunk specified. 13361 This can reduce the used download quota for some remotes by requesting 13362 only chunks from the remote that are actually read, at the cost of an 13363 increased number of requests. 13364 .PP 13365 These flags control the chunking: 13366 .IP 13367 .nf 13368 \f[C] 13369 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks (default 128M) 13370 --vfs-read-chunk-size-limit SizeSuffix Max chunk doubling size (default off) 13371 \f[R] 13372 .fi 13373 .PP 13374 Rclone will start reading a chunk of size 13375 \f[C]--vfs-read-chunk-size\f[R], and then double the size for each read. 13376 When \f[C]--vfs-read-chunk-size-limit\f[R] is specified, and greater 13377 than \f[C]--vfs-read-chunk-size\f[R], the chunk size for each open file 13378 will get doubled only until the specified value is reached. 13379 If the value is \[dq]off\[dq], which is the default, the limit is 13380 disabled and the chunk size will grow indefinitely. 13381 .PP 13382 With \f[C]--vfs-read-chunk-size 100M\f[R] and 13383 \f[C]--vfs-read-chunk-size-limit 0\f[R] the following parts will be 13384 downloaded: 0-100M, 100M-200M, 200M-300M, 300M-400M and so on. 13385 When \f[C]--vfs-read-chunk-size-limit 500M\f[R] is specified, the result 13386 would be 0-100M, 100M-300M, 300M-700M, 700M-1200M, 1200M-1700M and so 13387 on. 13388 .PP 13389 Setting \f[C]--vfs-read-chunk-size\f[R] to \f[C]0\f[R] or \[dq]off\[dq] 13390 disables chunked reading. 13391 .SS VFS Performance 13392 .PP 13393 These flags may be used to enable/disable features of the VFS for 13394 performance or other reasons. 13395 See also the chunked reading feature. 13396 .PP 13397 In particular S3 and Swift benefit hugely from the 13398 \f[C]--no-modtime\f[R] flag (or use \f[C]--use-server-modtime\f[R] for a 13399 slightly different effect) as each read of the modification time takes a 13400 transaction. 13401 .IP 13402 .nf 13403 \f[C] 13404 --no-checksum Don\[aq]t compare checksums on up/download. 13405 --no-modtime Don\[aq]t read/write the modification time (can speed things up). 13406 --no-seek Don\[aq]t allow seeking in files. 13407 --read-only Only allow read-only access. 13408 \f[R] 13409 .fi 13410 .PP 13411 Sometimes rclone is delivered reads or writes out of order. 13412 Rather than seeking rclone will wait a short time for the in sequence 13413 read or write to come in. 13414 These flags only come into effect when not using an on disk cache file. 13415 .IP 13416 .nf 13417 \f[C] 13418 --vfs-read-wait duration Time to wait for in-sequence read before seeking (default 20ms) 13419 --vfs-write-wait duration Time to wait for in-sequence write before giving error (default 1s) 13420 \f[R] 13421 .fi 13422 .PP 13423 When using VFS write caching (\f[C]--vfs-cache-mode\f[R] with value 13424 writes or full), the global flag \f[C]--transfers\f[R] can be set to 13425 adjust the number of parallel uploads of modified files from the cache 13426 (the related global flag \f[C]--checkers\f[R] has no effect on the VFS). 13427 .IP 13428 .nf 13429 \f[C] 13430 --transfers int Number of file transfers to run in parallel (default 4) 13431 \f[R] 13432 .fi 13433 .SS VFS Case Sensitivity 13434 .PP 13435 Linux file systems are case-sensitive: two files can differ only by 13436 case, and the exact case must be used when opening a file. 13437 .PP 13438 File systems in modern Windows are case-insensitive but case-preserving: 13439 although existing files can be opened using any case, the exact case 13440 used to create the file is preserved and available for programs to 13441 query. 13442 It is not allowed for two files in the same directory to differ only by 13443 case. 13444 .PP 13445 Usually file systems on macOS are case-insensitive. 13446 It is possible to make macOS file systems case-sensitive but that is not 13447 the default. 13448 .PP 13449 The \f[C]--vfs-case-insensitive\f[R] VFS flag controls how rclone 13450 handles these two cases. 13451 If its value is \[dq]false\[dq], rclone passes file names to the remote 13452 as-is. 13453 If the flag is \[dq]true\[dq] (or appears without a value on the command 13454 line), rclone may perform a \[dq]fixup\[dq] as explained below. 13455 .PP 13456 The user may specify a file name to open/delete/rename/etc with a case 13457 different than what is stored on the remote. 13458 If an argument refers to an existing file with exactly the same name, 13459 then the case of the existing file on the disk will be used. 13460 However, if a file name with exactly the same name is not found but a 13461 name differing only by case exists, rclone will transparently fixup the 13462 name. 13463 This fixup happens only when an existing file is requested. 13464 Case sensitivity of file names created anew by rclone is controlled by 13465 the underlying remote. 13466 .PP 13467 Note that case sensitivity of the operating system running rclone (the 13468 target) may differ from case sensitivity of a file system presented by 13469 rclone (the source). 13470 The flag controls whether \[dq]fixup\[dq] is performed to satisfy the 13471 target. 13472 .PP 13473 If the flag is not provided on the command line, then its default value 13474 depends on the operating system where rclone runs: \[dq]true\[dq] on 13475 Windows and macOS, \[dq]false\[dq] otherwise. 13476 If the flag is provided without a value, then it is \[dq]true\[dq]. 13477 .PP 13478 The \f[C]--no-unicode-normalization\f[R] flag controls whether a similar 13479 \[dq]fixup\[dq] is performed for filenames that differ but are 13480 canonically 13481 equivalent (https://en.wikipedia.org/wiki/Unicode_equivalence) with 13482 respect to unicode. 13483 Unicode normalization can be particularly helpful for users of macOS, 13484 which prefers form NFD instead of the NFC used by most other platforms. 13485 It is therefore highly recommended to keep the default of 13486 \f[C]false\f[R] on macOS, to avoid encoding compatibility issues. 13487 .PP 13488 In the (probably unlikely) event that a directory has multiple duplicate 13489 filenames after applying case and unicode normalization, the 13490 \f[C]--vfs-block-norm-dupes\f[R] flag allows hiding these duplicates. 13491 This comes with a performance tradeoff, as rclone will have to scan the 13492 entire directory for duplicates when listing a directory. 13493 For this reason, it is recommended to leave this disabled if not needed. 13494 However, macOS users may wish to consider using it, as otherwise, if a 13495 remote directory contains both NFC and NFD versions of the same 13496 filename, an odd situation will occur: both versions of the file will be 13497 visible in the mount, and both will appear to be editable, however, 13498 editing either version will actually result in only the NFD version 13499 getting edited under the hood. 13500 \f[C]--vfs-block- norm-dupes\f[R] prevents this confusion by detecting 13501 this scenario, hiding the duplicates, and logging an error, similar to 13502 how this is handled in \f[C]rclone sync\f[R]. 13503 .SS VFS Disk Options 13504 .PP 13505 This flag allows you to manually set the statistics about the filing 13506 system. 13507 It can be useful when those statistics cannot be read correctly 13508 automatically. 13509 .IP 13510 .nf 13511 \f[C] 13512 --vfs-disk-space-total-size Manually set the total disk space size (example: 256G, default: -1) 13513 \f[R] 13514 .fi 13515 .SS Alternate report of used bytes 13516 .PP 13517 Some backends, most notably S3, do not report the amount of bytes used. 13518 If you need this information to be available when running \f[C]df\f[R] 13519 on the filesystem, then pass the flag \f[C]--vfs-used-is-size\f[R] to 13520 rclone. 13521 With this flag set, instead of relying on the backend to report this 13522 information, rclone will scan the whole remote similar to 13523 \f[C]rclone size\f[R] and compute the total used space itself. 13524 .PP 13525 \f[I]WARNING.\f[R] Contrary to \f[C]rclone size\f[R], this flag ignores 13526 filters so that the result is accurate. 13527 However, this is very inefficient and may cost lots of API calls 13528 resulting in extra charges. 13529 Use it as a last resort and only with caching. 13530 .IP 13531 .nf 13532 \f[C] 13533 rclone serve s3 remote:path [flags] 13534 \f[R] 13535 .fi 13536 .SS Options 13537 .IP 13538 .nf 13539 \f[C] 13540 --addr stringArray IPaddress:Port or :Port to bind server to (default [127.0.0.1:8080]) 13541 --allow-origin string Origin which cross-domain request (CORS) can be executed from 13542 --auth-key stringArray Set key pair for v4 authorization: access_key_id,secret_access_key 13543 --baseurl string Prefix for URLs - leave blank for root 13544 --cert string TLS PEM key (concatenation of certificate and CA certificate) 13545 --client-ca string Client certificate authority to verify clients with 13546 --dir-cache-time Duration Time to cache directory entries for (default 5m0s) 13547 --dir-perms FileMode Directory permissions (default 0777) 13548 --etag-hash string Which hash to use for the ETag, or auto or blank for off (default \[dq]MD5\[dq]) 13549 --file-perms FileMode File permissions (default 0666) 13550 --force-path-style If true use path style access if false use virtual hosted style (default true) (default true) 13551 --gid uint32 Override the gid field set by the filesystem (not supported on Windows) (default 1000) 13552 -h, --help help for s3 13553 --key string TLS PEM Private key 13554 --max-header-bytes int Maximum size of request header (default 4096) 13555 --min-tls-version string Minimum TLS version that is acceptable (default \[dq]tls1.0\[dq]) 13556 --no-checksum Don\[aq]t compare checksums on up/download 13557 --no-cleanup Not to cleanup empty folder after object is deleted 13558 --no-modtime Don\[aq]t read/write the modification time (can speed things up) 13559 --no-seek Don\[aq]t allow seeking in files 13560 --poll-interval Duration Time to wait between polling for changes, must be smaller than dir-cache-time and only on supported remotes (set 0 to disable) (default 1m0s) 13561 --read-only Only allow read-only access 13562 --server-read-timeout Duration Timeout for server reading data (default 1h0m0s) 13563 --server-write-timeout Duration Timeout for server writing data (default 1h0m0s) 13564 --uid uint32 Override the uid field set by the filesystem (not supported on Windows) (default 1000) 13565 --umask int Override the permission bits set by the filesystem (not supported on Windows) (default 2) 13566 --vfs-block-norm-dupes If duplicate filenames exist in the same directory (after normalization), log an error and hide the duplicates (may have a performance cost) 13567 --vfs-cache-max-age Duration Max time since last access of objects in the cache (default 1h0m0s) 13568 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache (default off) 13569 --vfs-cache-min-free-space SizeSuffix Target minimum free space on the disk containing the cache (default off) 13570 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off) 13571 --vfs-cache-poll-interval Duration Interval to poll the cache for stale objects (default 1m0s) 13572 --vfs-case-insensitive If a file name not found, find a case insensitive match 13573 --vfs-disk-space-total-size SizeSuffix Specify the total space of disk (default off) 13574 --vfs-fast-fingerprint Use fast (less accurate) fingerprints for change detection 13575 --vfs-read-ahead SizeSuffix Extra read ahead over --buffer-size when using cache-mode full 13576 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks (default 128Mi) 13577 --vfs-read-chunk-size-limit SizeSuffix If greater than --vfs-read-chunk-size, double the chunk size after each chunk read, until the limit is reached (\[aq]off\[aq] is unlimited) (default off) 13578 --vfs-read-wait Duration Time to wait for in-sequence read before seeking (default 20ms) 13579 --vfs-refresh Refreshes the directory cache recursively in the background on start 13580 --vfs-used-is-size rclone size Use the rclone size algorithm for Used size 13581 --vfs-write-back Duration Time to writeback files after last use when using cache (default 5s) 13582 --vfs-write-wait Duration Time to wait for in-sequence write before giving error (default 1s) 13583 \f[R] 13584 .fi 13585 .SS Filter Options 13586 .PP 13587 Flags for filtering directory listings. 13588 .IP 13589 .nf 13590 \f[C] 13591 --delete-excluded Delete files on dest excluded from sync 13592 --exclude stringArray Exclude files matching pattern 13593 --exclude-from stringArray Read file exclude patterns from file (use - to read from stdin) 13594 --exclude-if-present stringArray Exclude directories if filename is present 13595 --files-from stringArray Read list of source-file names from file (use - to read from stdin) 13596 --files-from-raw stringArray Read list of source-file names from file without any processing of lines (use - to read from stdin) 13597 -f, --filter stringArray Add a file filtering rule 13598 --filter-from stringArray Read file filtering patterns from a file (use - to read from stdin) 13599 --ignore-case Ignore case in filters (case insensitive) 13600 --include stringArray Include files matching pattern 13601 --include-from stringArray Read file include patterns from file (use - to read from stdin) 13602 --max-age Duration Only transfer files younger than this in s or suffix ms|s|m|h|d|w|M|y (default off) 13603 --max-depth int If set limits the recursion depth to this (default -1) 13604 --max-size SizeSuffix Only transfer files smaller than this in KiB or suffix B|K|M|G|T|P (default off) 13605 --metadata-exclude stringArray Exclude metadatas matching pattern 13606 --metadata-exclude-from stringArray Read metadata exclude patterns from file (use - to read from stdin) 13607 --metadata-filter stringArray Add a metadata filtering rule 13608 --metadata-filter-from stringArray Read metadata filtering patterns from a file (use - to read from stdin) 13609 --metadata-include stringArray Include metadatas matching pattern 13610 --metadata-include-from stringArray Read metadata include patterns from file (use - to read from stdin) 13611 --min-age Duration Only transfer files older than this in s or suffix ms|s|m|h|d|w|M|y (default off) 13612 --min-size SizeSuffix Only transfer files bigger than this in KiB or suffix B|K|M|G|T|P (default off) 13613 \f[R] 13614 .fi 13615 .PP 13616 See the global flags page (https://rclone.org/flags/) for global options 13617 not listed here. 13618 .SH SEE ALSO 13619 .IP \[bu] 2 13620 rclone serve (https://rclone.org/commands/rclone_serve/) - Serve a 13621 remote over a protocol. 13622 .SH rclone serve sftp 13623 .PP 13624 Serve the remote over SFTP. 13625 .SS Synopsis 13626 .PP 13627 Run an SFTP server to serve a remote over SFTP. 13628 This can be used with an SFTP client or you can make a remote of type 13629 sftp to use with it. 13630 .PP 13631 You can use the filter flags (e.g. 13632 \f[C]--include\f[R], \f[C]--exclude\f[R]) to control what is served. 13633 .PP 13634 The server will respond to a small number of shell commands, mainly 13635 md5sum, sha1sum and df, which enable it to provide support for checksums 13636 and the about feature when accessed from an sftp remote. 13637 .PP 13638 Note that this server uses standard 32 KiB packet payload size, which 13639 means you must not configure the client to expect anything else, e.g. 13640 with the chunk_size (https://rclone.org/sftp/#sftp-chunk-size) option on 13641 an sftp remote. 13642 .PP 13643 The server will log errors. 13644 Use \f[C]-v\f[R] to see access logs. 13645 .PP 13646 \f[C]--bwlimit\f[R] will be respected for file transfers. 13647 Use \f[C]--stats\f[R] to control the stats printing. 13648 .PP 13649 You must provide some means of authentication, either with 13650 \f[C]--user\f[R]/\f[C]--pass\f[R], an authorized keys file (specify 13651 location with \f[C]--authorized-keys\f[R] - the default is the same as 13652 ssh), an \f[C]--auth-proxy\f[R], or set the \f[C]--no-auth\f[R] flag for 13653 no authentication when logging in. 13654 .PP 13655 If you don\[aq]t supply a host \f[C]--key\f[R] then rclone will generate 13656 rsa, ecdsa and ed25519 variants, and cache them for later use in 13657 rclone\[aq]s cache directory (see \f[C]rclone help flags cache-dir\f[R]) 13658 in the \[dq]serve-sftp\[dq] directory. 13659 .PP 13660 By default the server binds to localhost:2022 - if you want it to be 13661 reachable externally then supply \f[C]--addr :2022\f[R] for example. 13662 .PP 13663 Note that the default of \f[C]--vfs-cache-mode off\f[R] is fine for the 13664 rclone sftp backend, but it may not be with other SFTP clients. 13665 .PP 13666 If \f[C]--stdio\f[R] is specified, rclone will serve SFTP over stdio, 13667 which can be used with sshd via \[ti]/.ssh/authorized_keys, for example: 13668 .IP 13669 .nf 13670 \f[C] 13671 restrict,command=\[dq]rclone serve sftp --stdio ./photos\[dq] ssh-rsa ... 13672 \f[R] 13673 .fi 13674 .PP 13675 On the client you need to set \f[C]--transfers 1\f[R] when using 13676 \f[C]--stdio\f[R]. 13677 Otherwise multiple instances of the rclone server are started by OpenSSH 13678 which can lead to \[dq]corrupted on transfer\[dq] errors. 13679 This is the case because the client chooses indiscriminately which 13680 server to send commands to while the servers all have different views of 13681 the state of the filing system. 13682 .PP 13683 The \[dq]restrict\[dq] in authorized_keys prevents SHA1SUMs and MD5SUMs 13684 from being used. 13685 Omitting \[dq]restrict\[dq] and using \f[C]--sftp-path-override\f[R] to 13686 enable checksumming is possible but less secure and you could use the 13687 SFTP server provided by OpenSSH in this case. 13688 .SS VFS - Virtual File System 13689 .PP 13690 This command uses the VFS layer. 13691 This adapts the cloud storage objects that rclone uses into something 13692 which looks much more like a disk filing system. 13693 .PP 13694 Cloud storage objects have lots of properties which aren\[aq]t like disk 13695 files - you can\[aq]t extend them or write to the middle of them, so the 13696 VFS layer has to deal with that. 13697 Because there is no one right way of doing this there are various 13698 options explained below. 13699 .PP 13700 The VFS layer also implements a directory cache - this caches info about 13701 files and directories (but not the data) in memory. 13702 .SS VFS Directory Cache 13703 .PP 13704 Using the \f[C]--dir-cache-time\f[R] flag, you can control how long a 13705 directory should be considered up to date and not refreshed from the 13706 backend. 13707 Changes made through the VFS will appear immediately or invalidate the 13708 cache. 13709 .IP 13710 .nf 13711 \f[C] 13712 --dir-cache-time duration Time to cache directory entries for (default 5m0s) 13713 --poll-interval duration Time to wait between polling for changes. Must be smaller than dir-cache-time. Only on supported remotes. Set to 0 to disable (default 1m0s) 13714 \f[R] 13715 .fi 13716 .PP 13717 However, changes made directly on the cloud storage by the web interface 13718 or a different copy of rclone will only be picked up once the directory 13719 cache expires if the backend configured does not support polling for 13720 changes. 13721 If the backend supports polling, changes will be picked up within the 13722 polling interval. 13723 .PP 13724 You can send a \f[C]SIGHUP\f[R] signal to rclone for it to flush all 13725 directory caches, regardless of how old they are. 13726 Assuming only one rclone instance is running, you can reset the cache 13727 like this: 13728 .IP 13729 .nf 13730 \f[C] 13731 kill -SIGHUP $(pidof rclone) 13732 \f[R] 13733 .fi 13734 .PP 13735 If you configure rclone with a remote control then you can use rclone rc 13736 to flush the whole directory cache: 13737 .IP 13738 .nf 13739 \f[C] 13740 rclone rc vfs/forget 13741 \f[R] 13742 .fi 13743 .PP 13744 Or individual files or directories: 13745 .IP 13746 .nf 13747 \f[C] 13748 rclone rc vfs/forget file=path/to/file dir=path/to/dir 13749 \f[R] 13750 .fi 13751 .SS VFS File Buffering 13752 .PP 13753 The \f[C]--buffer-size\f[R] flag determines the amount of memory, that 13754 will be used to buffer data in advance. 13755 .PP 13756 Each open file will try to keep the specified amount of data in memory 13757 at all times. 13758 The buffered data is bound to one open file and won\[aq]t be shared. 13759 .PP 13760 This flag is a upper limit for the used memory per open file. 13761 The buffer will only use memory for data that is downloaded but not not 13762 yet read. 13763 If the buffer is empty, only a small amount of memory will be used. 13764 .PP 13765 The maximum memory used by rclone for buffering can be up to 13766 \f[C]--buffer-size * open files\f[R]. 13767 .SS VFS File Caching 13768 .PP 13769 These flags control the VFS file caching options. 13770 File caching is necessary to make the VFS layer appear compatible with a 13771 normal file system. 13772 It can be disabled at the cost of some compatibility. 13773 .PP 13774 For example you\[aq]ll need to enable VFS caching if you want to read 13775 and write simultaneously to a file. 13776 See below for more details. 13777 .PP 13778 Note that the VFS cache is separate from the cache backend and you may 13779 find that you need one or the other or both. 13780 .IP 13781 .nf 13782 \f[C] 13783 --cache-dir string Directory rclone will use for caching. 13784 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off) 13785 --vfs-cache-max-age duration Max time since last access of objects in the cache (default 1h0m0s) 13786 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache (default off) 13787 --vfs-cache-min-free-space SizeSuffix Target minimum free space on the disk containing the cache (default off) 13788 --vfs-cache-poll-interval duration Interval to poll the cache for stale objects (default 1m0s) 13789 --vfs-write-back duration Time to writeback files after last use when using cache (default 5s) 13790 \f[R] 13791 .fi 13792 .PP 13793 If run with \f[C]-vv\f[R] rclone will print the location of the file 13794 cache. 13795 The files are stored in the user cache file area which is OS dependent 13796 but can be controlled with \f[C]--cache-dir\f[R] or setting the 13797 appropriate environment variable. 13798 .PP 13799 The cache has 4 different modes selected by \f[C]--vfs-cache-mode\f[R]. 13800 The higher the cache mode the more compatible rclone becomes at the cost 13801 of using disk space. 13802 .PP 13803 Note that files are written back to the remote only when they are closed 13804 and if they haven\[aq]t been accessed for \f[C]--vfs-write-back\f[R] 13805 seconds. 13806 If rclone is quit or dies with files that haven\[aq]t been uploaded, 13807 these will be uploaded next time rclone is run with the same flags. 13808 .PP 13809 If using \f[C]--vfs-cache-max-size\f[R] or 13810 \f[C]--vfs-cache-min-free-size\f[R] note that the cache may exceed these 13811 quotas for two reasons. 13812 Firstly because it is only checked every 13813 \f[C]--vfs-cache-poll-interval\f[R]. 13814 Secondly because open files cannot be evicted from the cache. 13815 When \f[C]--vfs-cache-max-size\f[R] or 13816 \f[C]--vfs-cache-min-free-size\f[R] is exceeded, rclone will attempt to 13817 evict the least accessed files from the cache first. 13818 rclone will start with files that haven\[aq]t been accessed for the 13819 longest. 13820 This cache flushing strategy is efficient and more relevant files are 13821 likely to remain cached. 13822 .PP 13823 The \f[C]--vfs-cache-max-age\f[R] will evict files from the cache after 13824 the set time since last access has passed. 13825 The default value of 1 hour will start evicting files from cache that 13826 haven\[aq]t been accessed for 1 hour. 13827 When a cached file is accessed the 1 hour timer is reset to 0 and will 13828 wait for 1 more hour before evicting. 13829 Specify the time with standard notation, s, m, h, d, w . 13830 .PP 13831 You \f[B]should not\f[R] run two copies of rclone using the same VFS 13832 cache with the same or overlapping remotes if using 13833 \f[C]--vfs-cache-mode > off\f[R]. 13834 This can potentially cause data corruption if you do. 13835 You can work around this by giving each rclone its own cache hierarchy 13836 with \f[C]--cache-dir\f[R]. 13837 You don\[aq]t need to worry about this if the remotes in use don\[aq]t 13838 overlap. 13839 .SS --vfs-cache-mode off 13840 .PP 13841 In this mode (the default) the cache will read directly from the remote 13842 and write directly to the remote without caching anything on disk. 13843 .PP 13844 This will mean some operations are not possible 13845 .IP \[bu] 2 13846 Files can\[aq]t be opened for both read AND write 13847 .IP \[bu] 2 13848 Files opened for write can\[aq]t be seeked 13849 .IP \[bu] 2 13850 Existing files opened for write must have O_TRUNC set 13851 .IP \[bu] 2 13852 Files open for read with O_TRUNC will be opened write only 13853 .IP \[bu] 2 13854 Files open for write only will behave as if O_TRUNC was supplied 13855 .IP \[bu] 2 13856 Open modes O_APPEND, O_TRUNC are ignored 13857 .IP \[bu] 2 13858 If an upload fails it can\[aq]t be retried 13859 .SS --vfs-cache-mode minimal 13860 .PP 13861 This is very similar to \[dq]off\[dq] except that files opened for read 13862 AND write will be buffered to disk. 13863 This means that files opened for write will be a lot more compatible, 13864 but uses the minimal disk space. 13865 .PP 13866 These operations are not possible 13867 .IP \[bu] 2 13868 Files opened for write only can\[aq]t be seeked 13869 .IP \[bu] 2 13870 Existing files opened for write must have O_TRUNC set 13871 .IP \[bu] 2 13872 Files opened for write only will ignore O_APPEND, O_TRUNC 13873 .IP \[bu] 2 13874 If an upload fails it can\[aq]t be retried 13875 .SS --vfs-cache-mode writes 13876 .PP 13877 In this mode files opened for read only are still read directly from the 13878 remote, write only and read/write files are buffered to disk first. 13879 .PP 13880 This mode should support all normal file system operations. 13881 .PP 13882 If an upload fails it will be retried at exponentially increasing 13883 intervals up to 1 minute. 13884 .SS --vfs-cache-mode full 13885 .PP 13886 In this mode all reads and writes are buffered to and from disk. 13887 When data is read from the remote this is buffered to disk as well. 13888 .PP 13889 In this mode the files in the cache will be sparse files and rclone will 13890 keep track of which bits of the files it has downloaded. 13891 .PP 13892 So if an application only reads the starts of each file, then rclone 13893 will only buffer the start of the file. 13894 These files will appear to be their full size in the cache, but they 13895 will be sparse files with only the data that has been downloaded present 13896 in them. 13897 .PP 13898 This mode should support all normal file system operations and is 13899 otherwise identical to \f[C]--vfs-cache-mode\f[R] writes. 13900 .PP 13901 When reading a file rclone will read \f[C]--buffer-size\f[R] plus 13902 \f[C]--vfs-read-ahead\f[R] bytes ahead. 13903 The \f[C]--buffer-size\f[R] is buffered in memory whereas the 13904 \f[C]--vfs-read-ahead\f[R] is buffered on disk. 13905 .PP 13906 When using this mode it is recommended that \f[C]--buffer-size\f[R] is 13907 not set too large and \f[C]--vfs-read-ahead\f[R] is set large if 13908 required. 13909 .PP 13910 \f[B]IMPORTANT\f[R] not all file systems support sparse files. 13911 In particular FAT/exFAT do not. 13912 Rclone will perform very badly if the cache directory is on a filesystem 13913 which doesn\[aq]t support sparse files and it will log an ERROR message 13914 if one is detected. 13915 .SS Fingerprinting 13916 .PP 13917 Various parts of the VFS use fingerprinting to see if a local file copy 13918 has changed relative to a remote file. 13919 Fingerprints are made from: 13920 .IP \[bu] 2 13921 size 13922 .IP \[bu] 2 13923 modification time 13924 .IP \[bu] 2 13925 hash 13926 .PP 13927 where available on an object. 13928 .PP 13929 On some backends some of these attributes are slow to read (they take an 13930 extra API call per object, or extra work per object). 13931 .PP 13932 For example \f[C]hash\f[R] is slow with the \f[C]local\f[R] and 13933 \f[C]sftp\f[R] backends as they have to read the entire file and hash 13934 it, and \f[C]modtime\f[R] is slow with the \f[C]s3\f[R], 13935 \f[C]swift\f[R], \f[C]ftp\f[R] and \f[C]qinqstor\f[R] backends because 13936 they need to do an extra API call to fetch it. 13937 .PP 13938 If you use the \f[C]--vfs-fast-fingerprint\f[R] flag then rclone will 13939 not include the slow operations in the fingerprint. 13940 This makes the fingerprinting less accurate but much faster and will 13941 improve the opening time of cached files. 13942 .PP 13943 If you are running a vfs cache over \f[C]local\f[R], \f[C]s3\f[R] or 13944 \f[C]swift\f[R] backends then using this flag is recommended. 13945 .PP 13946 Note that if you change the value of this flag, the fingerprints of the 13947 files in the cache may be invalidated and the files will need to be 13948 downloaded again. 13949 .SS VFS Chunked Reading 13950 .PP 13951 When rclone reads files from a remote it reads them in chunks. 13952 This means that rather than requesting the whole file rclone reads the 13953 chunk specified. 13954 This can reduce the used download quota for some remotes by requesting 13955 only chunks from the remote that are actually read, at the cost of an 13956 increased number of requests. 13957 .PP 13958 These flags control the chunking: 13959 .IP 13960 .nf 13961 \f[C] 13962 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks (default 128M) 13963 --vfs-read-chunk-size-limit SizeSuffix Max chunk doubling size (default off) 13964 \f[R] 13965 .fi 13966 .PP 13967 Rclone will start reading a chunk of size 13968 \f[C]--vfs-read-chunk-size\f[R], and then double the size for each read. 13969 When \f[C]--vfs-read-chunk-size-limit\f[R] is specified, and greater 13970 than \f[C]--vfs-read-chunk-size\f[R], the chunk size for each open file 13971 will get doubled only until the specified value is reached. 13972 If the value is \[dq]off\[dq], which is the default, the limit is 13973 disabled and the chunk size will grow indefinitely. 13974 .PP 13975 With \f[C]--vfs-read-chunk-size 100M\f[R] and 13976 \f[C]--vfs-read-chunk-size-limit 0\f[R] the following parts will be 13977 downloaded: 0-100M, 100M-200M, 200M-300M, 300M-400M and so on. 13978 When \f[C]--vfs-read-chunk-size-limit 500M\f[R] is specified, the result 13979 would be 0-100M, 100M-300M, 300M-700M, 700M-1200M, 1200M-1700M and so 13980 on. 13981 .PP 13982 Setting \f[C]--vfs-read-chunk-size\f[R] to \f[C]0\f[R] or \[dq]off\[dq] 13983 disables chunked reading. 13984 .SS VFS Performance 13985 .PP 13986 These flags may be used to enable/disable features of the VFS for 13987 performance or other reasons. 13988 See also the chunked reading feature. 13989 .PP 13990 In particular S3 and Swift benefit hugely from the 13991 \f[C]--no-modtime\f[R] flag (or use \f[C]--use-server-modtime\f[R] for a 13992 slightly different effect) as each read of the modification time takes a 13993 transaction. 13994 .IP 13995 .nf 13996 \f[C] 13997 --no-checksum Don\[aq]t compare checksums on up/download. 13998 --no-modtime Don\[aq]t read/write the modification time (can speed things up). 13999 --no-seek Don\[aq]t allow seeking in files. 14000 --read-only Only allow read-only access. 14001 \f[R] 14002 .fi 14003 .PP 14004 Sometimes rclone is delivered reads or writes out of order. 14005 Rather than seeking rclone will wait a short time for the in sequence 14006 read or write to come in. 14007 These flags only come into effect when not using an on disk cache file. 14008 .IP 14009 .nf 14010 \f[C] 14011 --vfs-read-wait duration Time to wait for in-sequence read before seeking (default 20ms) 14012 --vfs-write-wait duration Time to wait for in-sequence write before giving error (default 1s) 14013 \f[R] 14014 .fi 14015 .PP 14016 When using VFS write caching (\f[C]--vfs-cache-mode\f[R] with value 14017 writes or full), the global flag \f[C]--transfers\f[R] can be set to 14018 adjust the number of parallel uploads of modified files from the cache 14019 (the related global flag \f[C]--checkers\f[R] has no effect on the VFS). 14020 .IP 14021 .nf 14022 \f[C] 14023 --transfers int Number of file transfers to run in parallel (default 4) 14024 \f[R] 14025 .fi 14026 .SS VFS Case Sensitivity 14027 .PP 14028 Linux file systems are case-sensitive: two files can differ only by 14029 case, and the exact case must be used when opening a file. 14030 .PP 14031 File systems in modern Windows are case-insensitive but case-preserving: 14032 although existing files can be opened using any case, the exact case 14033 used to create the file is preserved and available for programs to 14034 query. 14035 It is not allowed for two files in the same directory to differ only by 14036 case. 14037 .PP 14038 Usually file systems on macOS are case-insensitive. 14039 It is possible to make macOS file systems case-sensitive but that is not 14040 the default. 14041 .PP 14042 The \f[C]--vfs-case-insensitive\f[R] VFS flag controls how rclone 14043 handles these two cases. 14044 If its value is \[dq]false\[dq], rclone passes file names to the remote 14045 as-is. 14046 If the flag is \[dq]true\[dq] (or appears without a value on the command 14047 line), rclone may perform a \[dq]fixup\[dq] as explained below. 14048 .PP 14049 The user may specify a file name to open/delete/rename/etc with a case 14050 different than what is stored on the remote. 14051 If an argument refers to an existing file with exactly the same name, 14052 then the case of the existing file on the disk will be used. 14053 However, if a file name with exactly the same name is not found but a 14054 name differing only by case exists, rclone will transparently fixup the 14055 name. 14056 This fixup happens only when an existing file is requested. 14057 Case sensitivity of file names created anew by rclone is controlled by 14058 the underlying remote. 14059 .PP 14060 Note that case sensitivity of the operating system running rclone (the 14061 target) may differ from case sensitivity of a file system presented by 14062 rclone (the source). 14063 The flag controls whether \[dq]fixup\[dq] is performed to satisfy the 14064 target. 14065 .PP 14066 If the flag is not provided on the command line, then its default value 14067 depends on the operating system where rclone runs: \[dq]true\[dq] on 14068 Windows and macOS, \[dq]false\[dq] otherwise. 14069 If the flag is provided without a value, then it is \[dq]true\[dq]. 14070 .PP 14071 The \f[C]--no-unicode-normalization\f[R] flag controls whether a similar 14072 \[dq]fixup\[dq] is performed for filenames that differ but are 14073 canonically 14074 equivalent (https://en.wikipedia.org/wiki/Unicode_equivalence) with 14075 respect to unicode. 14076 Unicode normalization can be particularly helpful for users of macOS, 14077 which prefers form NFD instead of the NFC used by most other platforms. 14078 It is therefore highly recommended to keep the default of 14079 \f[C]false\f[R] on macOS, to avoid encoding compatibility issues. 14080 .PP 14081 In the (probably unlikely) event that a directory has multiple duplicate 14082 filenames after applying case and unicode normalization, the 14083 \f[C]--vfs-block-norm-dupes\f[R] flag allows hiding these duplicates. 14084 This comes with a performance tradeoff, as rclone will have to scan the 14085 entire directory for duplicates when listing a directory. 14086 For this reason, it is recommended to leave this disabled if not needed. 14087 However, macOS users may wish to consider using it, as otherwise, if a 14088 remote directory contains both NFC and NFD versions of the same 14089 filename, an odd situation will occur: both versions of the file will be 14090 visible in the mount, and both will appear to be editable, however, 14091 editing either version will actually result in only the NFD version 14092 getting edited under the hood. 14093 \f[C]--vfs-block- norm-dupes\f[R] prevents this confusion by detecting 14094 this scenario, hiding the duplicates, and logging an error, similar to 14095 how this is handled in \f[C]rclone sync\f[R]. 14096 .SS VFS Disk Options 14097 .PP 14098 This flag allows you to manually set the statistics about the filing 14099 system. 14100 It can be useful when those statistics cannot be read correctly 14101 automatically. 14102 .IP 14103 .nf 14104 \f[C] 14105 --vfs-disk-space-total-size Manually set the total disk space size (example: 256G, default: -1) 14106 \f[R] 14107 .fi 14108 .SS Alternate report of used bytes 14109 .PP 14110 Some backends, most notably S3, do not report the amount of bytes used. 14111 If you need this information to be available when running \f[C]df\f[R] 14112 on the filesystem, then pass the flag \f[C]--vfs-used-is-size\f[R] to 14113 rclone. 14114 With this flag set, instead of relying on the backend to report this 14115 information, rclone will scan the whole remote similar to 14116 \f[C]rclone size\f[R] and compute the total used space itself. 14117 .PP 14118 \f[I]WARNING.\f[R] Contrary to \f[C]rclone size\f[R], this flag ignores 14119 filters so that the result is accurate. 14120 However, this is very inefficient and may cost lots of API calls 14121 resulting in extra charges. 14122 Use it as a last resort and only with caching. 14123 .SS Auth Proxy 14124 .PP 14125 If you supply the parameter \f[C]--auth-proxy /path/to/program\f[R] then 14126 rclone will use that program to generate backends on the fly which then 14127 are used to authenticate incoming requests. 14128 This uses a simple JSON based protocol with input on STDIN and output on 14129 STDOUT. 14130 .PP 14131 \f[B]PLEASE NOTE:\f[R] \f[C]--auth-proxy\f[R] and 14132 \f[C]--authorized-keys\f[R] cannot be used together, if 14133 \f[C]--auth-proxy\f[R] is set the authorized keys option will be 14134 ignored. 14135 .PP 14136 There is an example program 14137 bin/test_proxy.py (https://github.com/artpar/artpar/blob/master/bin/test_proxy.py) 14138 in the rclone source code. 14139 .PP 14140 The program\[aq]s job is to take a \f[C]user\f[R] and \f[C]pass\f[R] on 14141 the input and turn those into the config for a backend on STDOUT in JSON 14142 format. 14143 This config will have any default parameters for the backend added, but 14144 it won\[aq]t use configuration from environment variables or command 14145 line options - it is the job of the proxy program to make a complete 14146 config. 14147 .PP 14148 This config generated must have this extra parameter - \f[C]_root\f[R] - 14149 root to use for the backend 14150 .PP 14151 And it may have this parameter - \f[C]_obscure\f[R] - comma separated 14152 strings for parameters to obscure 14153 .PP 14154 If password authentication was used by the client, input to the proxy 14155 process (on STDIN) would look similar to this: 14156 .IP 14157 .nf 14158 \f[C] 14159 { 14160 \[dq]user\[dq]: \[dq]me\[dq], 14161 \[dq]pass\[dq]: \[dq]mypassword\[dq] 14162 } 14163 \f[R] 14164 .fi 14165 .PP 14166 If public-key authentication was used by the client, input to the proxy 14167 process (on STDIN) would look similar to this: 14168 .IP 14169 .nf 14170 \f[C] 14171 { 14172 \[dq]user\[dq]: \[dq]me\[dq], 14173 \[dq]public_key\[dq]: \[dq]AAAAB3NzaC1yc2EAAAADAQABAAABAQDuwESFdAe14hVS6omeyX7edc...JQdf\[dq] 14174 } 14175 \f[R] 14176 .fi 14177 .PP 14178 And as an example return this on STDOUT 14179 .IP 14180 .nf 14181 \f[C] 14182 { 14183 \[dq]type\[dq]: \[dq]sftp\[dq], 14184 \[dq]_root\[dq]: \[dq]\[dq], 14185 \[dq]_obscure\[dq]: \[dq]pass\[dq], 14186 \[dq]user\[dq]: \[dq]me\[dq], 14187 \[dq]pass\[dq]: \[dq]mypassword\[dq], 14188 \[dq]host\[dq]: \[dq]sftp.example.com\[dq] 14189 } 14190 \f[R] 14191 .fi 14192 .PP 14193 This would mean that an SFTP backend would be created on the fly for the 14194 \f[C]user\f[R] and \f[C]pass\f[R]/\f[C]public_key\f[R] returned in the 14195 output to the host given. 14196 Note that since \f[C]_obscure\f[R] is set to \f[C]pass\f[R], rclone will 14197 obscure the \f[C]pass\f[R] parameter before creating the backend (which 14198 is required for sftp backends). 14199 .PP 14200 The program can manipulate the supplied \f[C]user\f[R] in any way, for 14201 example to make proxy to many different sftp backends, you could make 14202 the \f[C]user\f[R] be \f[C]user\[at]example.com\f[R] and then set the 14203 \f[C]host\f[R] to \f[C]example.com\f[R] in the output and the user to 14204 \f[C]user\f[R]. 14205 For security you\[aq]d probably want to restrict the \f[C]host\f[R] to a 14206 limited list. 14207 .PP 14208 Note that an internal cache is keyed on \f[C]user\f[R] so only use that 14209 for configuration, don\[aq]t use \f[C]pass\f[R] or \f[C]public_key\f[R]. 14210 This also means that if a user\[aq]s password or public-key is changed 14211 the cache will need to expire (which takes 5 mins) before it takes 14212 effect. 14213 .PP 14214 This can be used to build general purpose proxies to any kind of backend 14215 that rclone supports. 14216 .IP 14217 .nf 14218 \f[C] 14219 rclone serve sftp remote:path [flags] 14220 \f[R] 14221 .fi 14222 .SS Options 14223 .IP 14224 .nf 14225 \f[C] 14226 --addr string IPaddress:Port or :Port to bind server to (default \[dq]localhost:2022\[dq]) 14227 --auth-proxy string A program to use to create the backend from the auth 14228 --authorized-keys string Authorized keys file (default \[dq]\[ti]/.ssh/authorized_keys\[dq]) 14229 --dir-cache-time Duration Time to cache directory entries for (default 5m0s) 14230 --dir-perms FileMode Directory permissions (default 0777) 14231 --file-perms FileMode File permissions (default 0666) 14232 --gid uint32 Override the gid field set by the filesystem (not supported on Windows) (default 1000) 14233 -h, --help help for sftp 14234 --key stringArray SSH private host key file (Can be multi-valued, leave blank to auto generate) 14235 --no-auth Allow connections with no authentication if set 14236 --no-checksum Don\[aq]t compare checksums on up/download 14237 --no-modtime Don\[aq]t read/write the modification time (can speed things up) 14238 --no-seek Don\[aq]t allow seeking in files 14239 --pass string Password for authentication 14240 --poll-interval Duration Time to wait between polling for changes, must be smaller than dir-cache-time and only on supported remotes (set 0 to disable) (default 1m0s) 14241 --read-only Only allow read-only access 14242 --stdio Run an sftp server on stdin/stdout 14243 --uid uint32 Override the uid field set by the filesystem (not supported on Windows) (default 1000) 14244 --umask int Override the permission bits set by the filesystem (not supported on Windows) (default 2) 14245 --user string User name for authentication 14246 --vfs-block-norm-dupes If duplicate filenames exist in the same directory (after normalization), log an error and hide the duplicates (may have a performance cost) 14247 --vfs-cache-max-age Duration Max time since last access of objects in the cache (default 1h0m0s) 14248 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache (default off) 14249 --vfs-cache-min-free-space SizeSuffix Target minimum free space on the disk containing the cache (default off) 14250 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off) 14251 --vfs-cache-poll-interval Duration Interval to poll the cache for stale objects (default 1m0s) 14252 --vfs-case-insensitive If a file name not found, find a case insensitive match 14253 --vfs-disk-space-total-size SizeSuffix Specify the total space of disk (default off) 14254 --vfs-fast-fingerprint Use fast (less accurate) fingerprints for change detection 14255 --vfs-read-ahead SizeSuffix Extra read ahead over --buffer-size when using cache-mode full 14256 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks (default 128Mi) 14257 --vfs-read-chunk-size-limit SizeSuffix If greater than --vfs-read-chunk-size, double the chunk size after each chunk read, until the limit is reached (\[aq]off\[aq] is unlimited) (default off) 14258 --vfs-read-wait Duration Time to wait for in-sequence read before seeking (default 20ms) 14259 --vfs-refresh Refreshes the directory cache recursively in the background on start 14260 --vfs-used-is-size rclone size Use the rclone size algorithm for Used size 14261 --vfs-write-back Duration Time to writeback files after last use when using cache (default 5s) 14262 --vfs-write-wait Duration Time to wait for in-sequence write before giving error (default 1s) 14263 \f[R] 14264 .fi 14265 .SS Filter Options 14266 .PP 14267 Flags for filtering directory listings. 14268 .IP 14269 .nf 14270 \f[C] 14271 --delete-excluded Delete files on dest excluded from sync 14272 --exclude stringArray Exclude files matching pattern 14273 --exclude-from stringArray Read file exclude patterns from file (use - to read from stdin) 14274 --exclude-if-present stringArray Exclude directories if filename is present 14275 --files-from stringArray Read list of source-file names from file (use - to read from stdin) 14276 --files-from-raw stringArray Read list of source-file names from file without any processing of lines (use - to read from stdin) 14277 -f, --filter stringArray Add a file filtering rule 14278 --filter-from stringArray Read file filtering patterns from a file (use - to read from stdin) 14279 --ignore-case Ignore case in filters (case insensitive) 14280 --include stringArray Include files matching pattern 14281 --include-from stringArray Read file include patterns from file (use - to read from stdin) 14282 --max-age Duration Only transfer files younger than this in s or suffix ms|s|m|h|d|w|M|y (default off) 14283 --max-depth int If set limits the recursion depth to this (default -1) 14284 --max-size SizeSuffix Only transfer files smaller than this in KiB or suffix B|K|M|G|T|P (default off) 14285 --metadata-exclude stringArray Exclude metadatas matching pattern 14286 --metadata-exclude-from stringArray Read metadata exclude patterns from file (use - to read from stdin) 14287 --metadata-filter stringArray Add a metadata filtering rule 14288 --metadata-filter-from stringArray Read metadata filtering patterns from a file (use - to read from stdin) 14289 --metadata-include stringArray Include metadatas matching pattern 14290 --metadata-include-from stringArray Read metadata include patterns from file (use - to read from stdin) 14291 --min-age Duration Only transfer files older than this in s or suffix ms|s|m|h|d|w|M|y (default off) 14292 --min-size SizeSuffix Only transfer files bigger than this in KiB or suffix B|K|M|G|T|P (default off) 14293 \f[R] 14294 .fi 14295 .PP 14296 See the global flags page (https://rclone.org/flags/) for global options 14297 not listed here. 14298 .SH SEE ALSO 14299 .IP \[bu] 2 14300 rclone serve (https://rclone.org/commands/rclone_serve/) - Serve a 14301 remote over a protocol. 14302 .SH rclone serve webdav 14303 .PP 14304 Serve remote:path over WebDAV. 14305 .SS Synopsis 14306 .PP 14307 Run a basic WebDAV server to serve a remote over HTTP via the WebDAV 14308 protocol. 14309 This can be viewed with a WebDAV client, through a web browser, or you 14310 can make a remote of type WebDAV to read and write it. 14311 .SS WebDAV options 14312 .SS --etag-hash 14313 .PP 14314 This controls the ETag header. 14315 Without this flag the ETag will be based on the ModTime and Size of the 14316 object. 14317 .PP 14318 If this flag is set to \[dq]auto\[dq] then rclone will choose the first 14319 supported hash on the backend or you can use a named hash such as 14320 \[dq]MD5\[dq] or \[dq]SHA-1\[dq]. 14321 Use the hashsum (https://rclone.org/commands/rclone_hashsum/) command to 14322 see the full list. 14323 .SS Access WebDAV on Windows 14324 .PP 14325 WebDAV shared folder can be mapped as a drive on Windows, however the 14326 default settings prevent it. 14327 Windows will fail to connect to the server using insecure Basic 14328 authentication. 14329 It will not even display any login dialog. 14330 Windows requires SSL / HTTPS connection to be used with Basic. 14331 If you try to connect via Add Network Location Wizard you will get the 14332 following error: \[dq]The folder you entered does not appear to be 14333 valid. 14334 Please choose another\[dq]. 14335 However, you still can connect if you set the following registry key on 14336 a client machine: HKEY_LOCAL_MACHINEto 2. 14337 The BasicAuthLevel can be set to the following values: 0 - Basic 14338 authentication disabled 1 - Basic authentication enabled for SSL 14339 connections only 2 - Basic authentication enabled for SSL connections 14340 and for non-SSL connections If required, increase the 14341 FileSizeLimitInBytes to a higher value. 14342 Navigate to the Services interface, then restart the WebClient service. 14343 .SS Access Office applications on WebDAV 14344 .PP 14345 Navigate to following registry HKEY_CURRENT_USER[14.0/15.0/16.0] Create 14346 a new DWORD BasicAuthLevel with value 2. 14347 0 - Basic authentication disabled 1 - Basic authentication enabled for 14348 SSL connections only 2 - Basic authentication enabled for SSL and for 14349 non-SSL connections 14350 .PP 14351 https://learn.microsoft.com/en-us/office/troubleshoot/powerpoint/office-opens-blank-from-sharepoint 14352 .SS Server options 14353 .PP 14354 Use \f[C]--addr\f[R] to specify which IP address and port the server 14355 should listen on, eg \f[C]--addr 1.2.3.4:8000\f[R] or 14356 \f[C]--addr :8080\f[R] to listen to all IPs. 14357 By default it only listens on localhost. 14358 You can use port :0 to let the OS choose an available port. 14359 .PP 14360 If you set \f[C]--addr\f[R] to listen on a public or LAN accessible IP 14361 address then using Authentication is advised - see the next section for 14362 info. 14363 .PP 14364 You can use a unix socket by setting the url to 14365 \f[C]unix:///path/to/socket\f[R] or just by using an absolute path name. 14366 Note that unix sockets bypass the authentication - this is expected to 14367 be done with file system permissions. 14368 .PP 14369 \f[C]--addr\f[R] may be repeated to listen on multiple 14370 IPs/ports/sockets. 14371 .PP 14372 \f[C]--server-read-timeout\f[R] and \f[C]--server-write-timeout\f[R] can 14373 be used to control the timeouts on the server. 14374 Note that this is the total time for a transfer. 14375 .PP 14376 \f[C]--max-header-bytes\f[R] controls the maximum number of bytes the 14377 server will accept in the HTTP header. 14378 .PP 14379 \f[C]--baseurl\f[R] controls the URL prefix that rclone serves from. 14380 By default rclone will serve from the root. 14381 If you used \f[C]--baseurl \[dq]/rclone\[dq]\f[R] then rclone would 14382 serve from a URL starting with \[dq]/rclone/\[dq]. 14383 This is useful if you wish to proxy rclone serve. 14384 Rclone automatically inserts leading and trailing \[dq]/\[dq] on 14385 \f[C]--baseurl\f[R], so \f[C]--baseurl \[dq]rclone\[dq]\f[R], 14386 \f[C]--baseurl \[dq]/rclone\[dq]\f[R] and 14387 \f[C]--baseurl \[dq]/rclone/\[dq]\f[R] are all treated identically. 14388 .SS TLS (SSL) 14389 .PP 14390 By default this will serve over http. 14391 If you want you can serve over https. 14392 You will need to supply the \f[C]--cert\f[R] and \f[C]--key\f[R] flags. 14393 If you wish to do client side certificate validation then you will need 14394 to supply \f[C]--client-ca\f[R] also. 14395 .PP 14396 \f[C]--cert\f[R] should be a either a PEM encoded certificate or a 14397 concatenation of that with the CA certificate. 14398 \f[C]--key\f[R] should be the PEM encoded private key and 14399 \f[C]--client-ca\f[R] should be the PEM encoded client certificate 14400 authority certificate. 14401 .PP 14402 --min-tls-version is minimum TLS version that is acceptable. 14403 Valid values are \[dq]tls1.0\[dq], \[dq]tls1.1\[dq], \[dq]tls1.2\[dq] 14404 and \[dq]tls1.3\[dq] (default \[dq]tls1.0\[dq]). 14405 .SS Template 14406 .PP 14407 \f[C]--template\f[R] allows a user to specify a custom markup template 14408 for HTTP and WebDAV serve functions. 14409 The server exports the following markup to be used within the template 14410 to server pages: 14411 .PP 14412 .TS 14413 tab(@); 14414 lw(35.0n) lw(35.0n). 14415 T{ 14416 Parameter 14417 T}@T{ 14418 Description 14419 T} 14420 _ 14421 T{ 14422 \&.Name 14423 T}@T{ 14424 The full path of a file/directory. 14425 T} 14426 T{ 14427 \&.Title 14428 T}@T{ 14429 Directory listing of .Name 14430 T} 14431 T{ 14432 \&.Sort 14433 T}@T{ 14434 The current sort used. 14435 This is changeable via ?sort= parameter 14436 T} 14437 T{ 14438 T}@T{ 14439 Sort Options: namedirfirst,name,size,time (default namedirfirst) 14440 T} 14441 T{ 14442 \&.Order 14443 T}@T{ 14444 The current ordering used. 14445 This is changeable via ?order= parameter 14446 T} 14447 T{ 14448 T}@T{ 14449 Order Options: asc,desc (default asc) 14450 T} 14451 T{ 14452 \&.Query 14453 T}@T{ 14454 Currently unused. 14455 T} 14456 T{ 14457 \&.Breadcrumb 14458 T}@T{ 14459 Allows for creating a relative navigation 14460 T} 14461 T{ 14462 -- .Link 14463 T}@T{ 14464 The relative to the root link of the Text. 14465 T} 14466 T{ 14467 -- .Text 14468 T}@T{ 14469 The Name of the directory. 14470 T} 14471 T{ 14472 \&.Entries 14473 T}@T{ 14474 Information about a specific file/directory. 14475 T} 14476 T{ 14477 -- .URL 14478 T}@T{ 14479 The \[aq]url\[aq] of an entry. 14480 T} 14481 T{ 14482 -- .Leaf 14483 T}@T{ 14484 Currently same as \[aq]URL\[aq] but intended to be \[aq]just\[aq] the 14485 name. 14486 T} 14487 T{ 14488 -- .IsDir 14489 T}@T{ 14490 Boolean for if an entry is a directory or not. 14491 T} 14492 T{ 14493 -- .Size 14494 T}@T{ 14495 Size in Bytes of the entry. 14496 T} 14497 T{ 14498 -- .ModTime 14499 T}@T{ 14500 The UTC timestamp of an entry. 14501 T} 14502 .TE 14503 .PP 14504 The server also makes the following functions available so that they can 14505 be used within the template. 14506 These functions help extend the options for dynamic rendering of HTML. 14507 They can be used to render HTML based on specific conditions. 14508 .PP 14509 .TS 14510 tab(@); 14511 lw(35.0n) lw(35.0n). 14512 T{ 14513 Function 14514 T}@T{ 14515 Description 14516 T} 14517 _ 14518 T{ 14519 afterEpoch 14520 T}@T{ 14521 Returns the time since the epoch for the given time. 14522 T} 14523 T{ 14524 contains 14525 T}@T{ 14526 Checks whether a given substring is present or not in a given string. 14527 T} 14528 T{ 14529 hasPrefix 14530 T}@T{ 14531 Checks whether the given string begins with the specified prefix. 14532 T} 14533 T{ 14534 hasSuffix 14535 T}@T{ 14536 Checks whether the given string end with the specified suffix. 14537 T} 14538 .TE 14539 .SS Authentication 14540 .PP 14541 By default this will serve files without needing a login. 14542 .PP 14543 You can either use an htpasswd file which can take lots of users, or set 14544 a single username and password with the \f[C]--user\f[R] and 14545 \f[C]--pass\f[R] flags. 14546 .PP 14547 If no static users are configured by either of the above methods, and 14548 client certificates are required by the \f[C]--client-ca\f[R] flag 14549 passed to the server, the client certificate common name will be 14550 considered as the username. 14551 .PP 14552 Use \f[C]--htpasswd /path/to/htpasswd\f[R] to provide an htpasswd file. 14553 This is in standard apache format and supports MD5, SHA1 and BCrypt for 14554 basic authentication. 14555 Bcrypt is recommended. 14556 .PP 14557 To create an htpasswd file: 14558 .IP 14559 .nf 14560 \f[C] 14561 touch htpasswd 14562 htpasswd -B htpasswd user 14563 htpasswd -B htpasswd anotherUser 14564 \f[R] 14565 .fi 14566 .PP 14567 The password file can be updated while rclone is running. 14568 .PP 14569 Use \f[C]--realm\f[R] to set the authentication realm. 14570 .PP 14571 Use \f[C]--salt\f[R] to change the password hashing salt from the 14572 default. 14573 ## VFS - Virtual File System 14574 .PP 14575 This command uses the VFS layer. 14576 This adapts the cloud storage objects that rclone uses into something 14577 which looks much more like a disk filing system. 14578 .PP 14579 Cloud storage objects have lots of properties which aren\[aq]t like disk 14580 files - you can\[aq]t extend them or write to the middle of them, so the 14581 VFS layer has to deal with that. 14582 Because there is no one right way of doing this there are various 14583 options explained below. 14584 .PP 14585 The VFS layer also implements a directory cache - this caches info about 14586 files and directories (but not the data) in memory. 14587 .SS VFS Directory Cache 14588 .PP 14589 Using the \f[C]--dir-cache-time\f[R] flag, you can control how long a 14590 directory should be considered up to date and not refreshed from the 14591 backend. 14592 Changes made through the VFS will appear immediately or invalidate the 14593 cache. 14594 .IP 14595 .nf 14596 \f[C] 14597 --dir-cache-time duration Time to cache directory entries for (default 5m0s) 14598 --poll-interval duration Time to wait between polling for changes. Must be smaller than dir-cache-time. Only on supported remotes. Set to 0 to disable (default 1m0s) 14599 \f[R] 14600 .fi 14601 .PP 14602 However, changes made directly on the cloud storage by the web interface 14603 or a different copy of rclone will only be picked up once the directory 14604 cache expires if the backend configured does not support polling for 14605 changes. 14606 If the backend supports polling, changes will be picked up within the 14607 polling interval. 14608 .PP 14609 You can send a \f[C]SIGHUP\f[R] signal to rclone for it to flush all 14610 directory caches, regardless of how old they are. 14611 Assuming only one rclone instance is running, you can reset the cache 14612 like this: 14613 .IP 14614 .nf 14615 \f[C] 14616 kill -SIGHUP $(pidof rclone) 14617 \f[R] 14618 .fi 14619 .PP 14620 If you configure rclone with a remote control then you can use rclone rc 14621 to flush the whole directory cache: 14622 .IP 14623 .nf 14624 \f[C] 14625 rclone rc vfs/forget 14626 \f[R] 14627 .fi 14628 .PP 14629 Or individual files or directories: 14630 .IP 14631 .nf 14632 \f[C] 14633 rclone rc vfs/forget file=path/to/file dir=path/to/dir 14634 \f[R] 14635 .fi 14636 .SS VFS File Buffering 14637 .PP 14638 The \f[C]--buffer-size\f[R] flag determines the amount of memory, that 14639 will be used to buffer data in advance. 14640 .PP 14641 Each open file will try to keep the specified amount of data in memory 14642 at all times. 14643 The buffered data is bound to one open file and won\[aq]t be shared. 14644 .PP 14645 This flag is a upper limit for the used memory per open file. 14646 The buffer will only use memory for data that is downloaded but not not 14647 yet read. 14648 If the buffer is empty, only a small amount of memory will be used. 14649 .PP 14650 The maximum memory used by rclone for buffering can be up to 14651 \f[C]--buffer-size * open files\f[R]. 14652 .SS VFS File Caching 14653 .PP 14654 These flags control the VFS file caching options. 14655 File caching is necessary to make the VFS layer appear compatible with a 14656 normal file system. 14657 It can be disabled at the cost of some compatibility. 14658 .PP 14659 For example you\[aq]ll need to enable VFS caching if you want to read 14660 and write simultaneously to a file. 14661 See below for more details. 14662 .PP 14663 Note that the VFS cache is separate from the cache backend and you may 14664 find that you need one or the other or both. 14665 .IP 14666 .nf 14667 \f[C] 14668 --cache-dir string Directory rclone will use for caching. 14669 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off) 14670 --vfs-cache-max-age duration Max time since last access of objects in the cache (default 1h0m0s) 14671 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache (default off) 14672 --vfs-cache-min-free-space SizeSuffix Target minimum free space on the disk containing the cache (default off) 14673 --vfs-cache-poll-interval duration Interval to poll the cache for stale objects (default 1m0s) 14674 --vfs-write-back duration Time to writeback files after last use when using cache (default 5s) 14675 \f[R] 14676 .fi 14677 .PP 14678 If run with \f[C]-vv\f[R] rclone will print the location of the file 14679 cache. 14680 The files are stored in the user cache file area which is OS dependent 14681 but can be controlled with \f[C]--cache-dir\f[R] or setting the 14682 appropriate environment variable. 14683 .PP 14684 The cache has 4 different modes selected by \f[C]--vfs-cache-mode\f[R]. 14685 The higher the cache mode the more compatible rclone becomes at the cost 14686 of using disk space. 14687 .PP 14688 Note that files are written back to the remote only when they are closed 14689 and if they haven\[aq]t been accessed for \f[C]--vfs-write-back\f[R] 14690 seconds. 14691 If rclone is quit or dies with files that haven\[aq]t been uploaded, 14692 these will be uploaded next time rclone is run with the same flags. 14693 .PP 14694 If using \f[C]--vfs-cache-max-size\f[R] or 14695 \f[C]--vfs-cache-min-free-size\f[R] note that the cache may exceed these 14696 quotas for two reasons. 14697 Firstly because it is only checked every 14698 \f[C]--vfs-cache-poll-interval\f[R]. 14699 Secondly because open files cannot be evicted from the cache. 14700 When \f[C]--vfs-cache-max-size\f[R] or 14701 \f[C]--vfs-cache-min-free-size\f[R] is exceeded, rclone will attempt to 14702 evict the least accessed files from the cache first. 14703 rclone will start with files that haven\[aq]t been accessed for the 14704 longest. 14705 This cache flushing strategy is efficient and more relevant files are 14706 likely to remain cached. 14707 .PP 14708 The \f[C]--vfs-cache-max-age\f[R] will evict files from the cache after 14709 the set time since last access has passed. 14710 The default value of 1 hour will start evicting files from cache that 14711 haven\[aq]t been accessed for 1 hour. 14712 When a cached file is accessed the 1 hour timer is reset to 0 and will 14713 wait for 1 more hour before evicting. 14714 Specify the time with standard notation, s, m, h, d, w . 14715 .PP 14716 You \f[B]should not\f[R] run two copies of rclone using the same VFS 14717 cache with the same or overlapping remotes if using 14718 \f[C]--vfs-cache-mode > off\f[R]. 14719 This can potentially cause data corruption if you do. 14720 You can work around this by giving each rclone its own cache hierarchy 14721 with \f[C]--cache-dir\f[R]. 14722 You don\[aq]t need to worry about this if the remotes in use don\[aq]t 14723 overlap. 14724 .SS --vfs-cache-mode off 14725 .PP 14726 In this mode (the default) the cache will read directly from the remote 14727 and write directly to the remote without caching anything on disk. 14728 .PP 14729 This will mean some operations are not possible 14730 .IP \[bu] 2 14731 Files can\[aq]t be opened for both read AND write 14732 .IP \[bu] 2 14733 Files opened for write can\[aq]t be seeked 14734 .IP \[bu] 2 14735 Existing files opened for write must have O_TRUNC set 14736 .IP \[bu] 2 14737 Files open for read with O_TRUNC will be opened write only 14738 .IP \[bu] 2 14739 Files open for write only will behave as if O_TRUNC was supplied 14740 .IP \[bu] 2 14741 Open modes O_APPEND, O_TRUNC are ignored 14742 .IP \[bu] 2 14743 If an upload fails it can\[aq]t be retried 14744 .SS --vfs-cache-mode minimal 14745 .PP 14746 This is very similar to \[dq]off\[dq] except that files opened for read 14747 AND write will be buffered to disk. 14748 This means that files opened for write will be a lot more compatible, 14749 but uses the minimal disk space. 14750 .PP 14751 These operations are not possible 14752 .IP \[bu] 2 14753 Files opened for write only can\[aq]t be seeked 14754 .IP \[bu] 2 14755 Existing files opened for write must have O_TRUNC set 14756 .IP \[bu] 2 14757 Files opened for write only will ignore O_APPEND, O_TRUNC 14758 .IP \[bu] 2 14759 If an upload fails it can\[aq]t be retried 14760 .SS --vfs-cache-mode writes 14761 .PP 14762 In this mode files opened for read only are still read directly from the 14763 remote, write only and read/write files are buffered to disk first. 14764 .PP 14765 This mode should support all normal file system operations. 14766 .PP 14767 If an upload fails it will be retried at exponentially increasing 14768 intervals up to 1 minute. 14769 .SS --vfs-cache-mode full 14770 .PP 14771 In this mode all reads and writes are buffered to and from disk. 14772 When data is read from the remote this is buffered to disk as well. 14773 .PP 14774 In this mode the files in the cache will be sparse files and rclone will 14775 keep track of which bits of the files it has downloaded. 14776 .PP 14777 So if an application only reads the starts of each file, then rclone 14778 will only buffer the start of the file. 14779 These files will appear to be their full size in the cache, but they 14780 will be sparse files with only the data that has been downloaded present 14781 in them. 14782 .PP 14783 This mode should support all normal file system operations and is 14784 otherwise identical to \f[C]--vfs-cache-mode\f[R] writes. 14785 .PP 14786 When reading a file rclone will read \f[C]--buffer-size\f[R] plus 14787 \f[C]--vfs-read-ahead\f[R] bytes ahead. 14788 The \f[C]--buffer-size\f[R] is buffered in memory whereas the 14789 \f[C]--vfs-read-ahead\f[R] is buffered on disk. 14790 .PP 14791 When using this mode it is recommended that \f[C]--buffer-size\f[R] is 14792 not set too large and \f[C]--vfs-read-ahead\f[R] is set large if 14793 required. 14794 .PP 14795 \f[B]IMPORTANT\f[R] not all file systems support sparse files. 14796 In particular FAT/exFAT do not. 14797 Rclone will perform very badly if the cache directory is on a filesystem 14798 which doesn\[aq]t support sparse files and it will log an ERROR message 14799 if one is detected. 14800 .SS Fingerprinting 14801 .PP 14802 Various parts of the VFS use fingerprinting to see if a local file copy 14803 has changed relative to a remote file. 14804 Fingerprints are made from: 14805 .IP \[bu] 2 14806 size 14807 .IP \[bu] 2 14808 modification time 14809 .IP \[bu] 2 14810 hash 14811 .PP 14812 where available on an object. 14813 .PP 14814 On some backends some of these attributes are slow to read (they take an 14815 extra API call per object, or extra work per object). 14816 .PP 14817 For example \f[C]hash\f[R] is slow with the \f[C]local\f[R] and 14818 \f[C]sftp\f[R] backends as they have to read the entire file and hash 14819 it, and \f[C]modtime\f[R] is slow with the \f[C]s3\f[R], 14820 \f[C]swift\f[R], \f[C]ftp\f[R] and \f[C]qinqstor\f[R] backends because 14821 they need to do an extra API call to fetch it. 14822 .PP 14823 If you use the \f[C]--vfs-fast-fingerprint\f[R] flag then rclone will 14824 not include the slow operations in the fingerprint. 14825 This makes the fingerprinting less accurate but much faster and will 14826 improve the opening time of cached files. 14827 .PP 14828 If you are running a vfs cache over \f[C]local\f[R], \f[C]s3\f[R] or 14829 \f[C]swift\f[R] backends then using this flag is recommended. 14830 .PP 14831 Note that if you change the value of this flag, the fingerprints of the 14832 files in the cache may be invalidated and the files will need to be 14833 downloaded again. 14834 .SS VFS Chunked Reading 14835 .PP 14836 When rclone reads files from a remote it reads them in chunks. 14837 This means that rather than requesting the whole file rclone reads the 14838 chunk specified. 14839 This can reduce the used download quota for some remotes by requesting 14840 only chunks from the remote that are actually read, at the cost of an 14841 increased number of requests. 14842 .PP 14843 These flags control the chunking: 14844 .IP 14845 .nf 14846 \f[C] 14847 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks (default 128M) 14848 --vfs-read-chunk-size-limit SizeSuffix Max chunk doubling size (default off) 14849 \f[R] 14850 .fi 14851 .PP 14852 Rclone will start reading a chunk of size 14853 \f[C]--vfs-read-chunk-size\f[R], and then double the size for each read. 14854 When \f[C]--vfs-read-chunk-size-limit\f[R] is specified, and greater 14855 than \f[C]--vfs-read-chunk-size\f[R], the chunk size for each open file 14856 will get doubled only until the specified value is reached. 14857 If the value is \[dq]off\[dq], which is the default, the limit is 14858 disabled and the chunk size will grow indefinitely. 14859 .PP 14860 With \f[C]--vfs-read-chunk-size 100M\f[R] and 14861 \f[C]--vfs-read-chunk-size-limit 0\f[R] the following parts will be 14862 downloaded: 0-100M, 100M-200M, 200M-300M, 300M-400M and so on. 14863 When \f[C]--vfs-read-chunk-size-limit 500M\f[R] is specified, the result 14864 would be 0-100M, 100M-300M, 300M-700M, 700M-1200M, 1200M-1700M and so 14865 on. 14866 .PP 14867 Setting \f[C]--vfs-read-chunk-size\f[R] to \f[C]0\f[R] or \[dq]off\[dq] 14868 disables chunked reading. 14869 .SS VFS Performance 14870 .PP 14871 These flags may be used to enable/disable features of the VFS for 14872 performance or other reasons. 14873 See also the chunked reading feature. 14874 .PP 14875 In particular S3 and Swift benefit hugely from the 14876 \f[C]--no-modtime\f[R] flag (or use \f[C]--use-server-modtime\f[R] for a 14877 slightly different effect) as each read of the modification time takes a 14878 transaction. 14879 .IP 14880 .nf 14881 \f[C] 14882 --no-checksum Don\[aq]t compare checksums on up/download. 14883 --no-modtime Don\[aq]t read/write the modification time (can speed things up). 14884 --no-seek Don\[aq]t allow seeking in files. 14885 --read-only Only allow read-only access. 14886 \f[R] 14887 .fi 14888 .PP 14889 Sometimes rclone is delivered reads or writes out of order. 14890 Rather than seeking rclone will wait a short time for the in sequence 14891 read or write to come in. 14892 These flags only come into effect when not using an on disk cache file. 14893 .IP 14894 .nf 14895 \f[C] 14896 --vfs-read-wait duration Time to wait for in-sequence read before seeking (default 20ms) 14897 --vfs-write-wait duration Time to wait for in-sequence write before giving error (default 1s) 14898 \f[R] 14899 .fi 14900 .PP 14901 When using VFS write caching (\f[C]--vfs-cache-mode\f[R] with value 14902 writes or full), the global flag \f[C]--transfers\f[R] can be set to 14903 adjust the number of parallel uploads of modified files from the cache 14904 (the related global flag \f[C]--checkers\f[R] has no effect on the VFS). 14905 .IP 14906 .nf 14907 \f[C] 14908 --transfers int Number of file transfers to run in parallel (default 4) 14909 \f[R] 14910 .fi 14911 .SS VFS Case Sensitivity 14912 .PP 14913 Linux file systems are case-sensitive: two files can differ only by 14914 case, and the exact case must be used when opening a file. 14915 .PP 14916 File systems in modern Windows are case-insensitive but case-preserving: 14917 although existing files can be opened using any case, the exact case 14918 used to create the file is preserved and available for programs to 14919 query. 14920 It is not allowed for two files in the same directory to differ only by 14921 case. 14922 .PP 14923 Usually file systems on macOS are case-insensitive. 14924 It is possible to make macOS file systems case-sensitive but that is not 14925 the default. 14926 .PP 14927 The \f[C]--vfs-case-insensitive\f[R] VFS flag controls how rclone 14928 handles these two cases. 14929 If its value is \[dq]false\[dq], rclone passes file names to the remote 14930 as-is. 14931 If the flag is \[dq]true\[dq] (or appears without a value on the command 14932 line), rclone may perform a \[dq]fixup\[dq] as explained below. 14933 .PP 14934 The user may specify a file name to open/delete/rename/etc with a case 14935 different than what is stored on the remote. 14936 If an argument refers to an existing file with exactly the same name, 14937 then the case of the existing file on the disk will be used. 14938 However, if a file name with exactly the same name is not found but a 14939 name differing only by case exists, rclone will transparently fixup the 14940 name. 14941 This fixup happens only when an existing file is requested. 14942 Case sensitivity of file names created anew by rclone is controlled by 14943 the underlying remote. 14944 .PP 14945 Note that case sensitivity of the operating system running rclone (the 14946 target) may differ from case sensitivity of a file system presented by 14947 rclone (the source). 14948 The flag controls whether \[dq]fixup\[dq] is performed to satisfy the 14949 target. 14950 .PP 14951 If the flag is not provided on the command line, then its default value 14952 depends on the operating system where rclone runs: \[dq]true\[dq] on 14953 Windows and macOS, \[dq]false\[dq] otherwise. 14954 If the flag is provided without a value, then it is \[dq]true\[dq]. 14955 .PP 14956 The \f[C]--no-unicode-normalization\f[R] flag controls whether a similar 14957 \[dq]fixup\[dq] is performed for filenames that differ but are 14958 canonically 14959 equivalent (https://en.wikipedia.org/wiki/Unicode_equivalence) with 14960 respect to unicode. 14961 Unicode normalization can be particularly helpful for users of macOS, 14962 which prefers form NFD instead of the NFC used by most other platforms. 14963 It is therefore highly recommended to keep the default of 14964 \f[C]false\f[R] on macOS, to avoid encoding compatibility issues. 14965 .PP 14966 In the (probably unlikely) event that a directory has multiple duplicate 14967 filenames after applying case and unicode normalization, the 14968 \f[C]--vfs-block-norm-dupes\f[R] flag allows hiding these duplicates. 14969 This comes with a performance tradeoff, as rclone will have to scan the 14970 entire directory for duplicates when listing a directory. 14971 For this reason, it is recommended to leave this disabled if not needed. 14972 However, macOS users may wish to consider using it, as otherwise, if a 14973 remote directory contains both NFC and NFD versions of the same 14974 filename, an odd situation will occur: both versions of the file will be 14975 visible in the mount, and both will appear to be editable, however, 14976 editing either version will actually result in only the NFD version 14977 getting edited under the hood. 14978 \f[C]--vfs-block- norm-dupes\f[R] prevents this confusion by detecting 14979 this scenario, hiding the duplicates, and logging an error, similar to 14980 how this is handled in \f[C]rclone sync\f[R]. 14981 .SS VFS Disk Options 14982 .PP 14983 This flag allows you to manually set the statistics about the filing 14984 system. 14985 It can be useful when those statistics cannot be read correctly 14986 automatically. 14987 .IP 14988 .nf 14989 \f[C] 14990 --vfs-disk-space-total-size Manually set the total disk space size (example: 256G, default: -1) 14991 \f[R] 14992 .fi 14993 .SS Alternate report of used bytes 14994 .PP 14995 Some backends, most notably S3, do not report the amount of bytes used. 14996 If you need this information to be available when running \f[C]df\f[R] 14997 on the filesystem, then pass the flag \f[C]--vfs-used-is-size\f[R] to 14998 rclone. 14999 With this flag set, instead of relying on the backend to report this 15000 information, rclone will scan the whole remote similar to 15001 \f[C]rclone size\f[R] and compute the total used space itself. 15002 .PP 15003 \f[I]WARNING.\f[R] Contrary to \f[C]rclone size\f[R], this flag ignores 15004 filters so that the result is accurate. 15005 However, this is very inefficient and may cost lots of API calls 15006 resulting in extra charges. 15007 Use it as a last resort and only with caching. 15008 .SS Auth Proxy 15009 .PP 15010 If you supply the parameter \f[C]--auth-proxy /path/to/program\f[R] then 15011 rclone will use that program to generate backends on the fly which then 15012 are used to authenticate incoming requests. 15013 This uses a simple JSON based protocol with input on STDIN and output on 15014 STDOUT. 15015 .PP 15016 \f[B]PLEASE NOTE:\f[R] \f[C]--auth-proxy\f[R] and 15017 \f[C]--authorized-keys\f[R] cannot be used together, if 15018 \f[C]--auth-proxy\f[R] is set the authorized keys option will be 15019 ignored. 15020 .PP 15021 There is an example program 15022 bin/test_proxy.py (https://github.com/artpar/artpar/blob/master/bin/test_proxy.py) 15023 in the rclone source code. 15024 .PP 15025 The program\[aq]s job is to take a \f[C]user\f[R] and \f[C]pass\f[R] on 15026 the input and turn those into the config for a backend on STDOUT in JSON 15027 format. 15028 This config will have any default parameters for the backend added, but 15029 it won\[aq]t use configuration from environment variables or command 15030 line options - it is the job of the proxy program to make a complete 15031 config. 15032 .PP 15033 This config generated must have this extra parameter - \f[C]_root\f[R] - 15034 root to use for the backend 15035 .PP 15036 And it may have this parameter - \f[C]_obscure\f[R] - comma separated 15037 strings for parameters to obscure 15038 .PP 15039 If password authentication was used by the client, input to the proxy 15040 process (on STDIN) would look similar to this: 15041 .IP 15042 .nf 15043 \f[C] 15044 { 15045 \[dq]user\[dq]: \[dq]me\[dq], 15046 \[dq]pass\[dq]: \[dq]mypassword\[dq] 15047 } 15048 \f[R] 15049 .fi 15050 .PP 15051 If public-key authentication was used by the client, input to the proxy 15052 process (on STDIN) would look similar to this: 15053 .IP 15054 .nf 15055 \f[C] 15056 { 15057 \[dq]user\[dq]: \[dq]me\[dq], 15058 \[dq]public_key\[dq]: \[dq]AAAAB3NzaC1yc2EAAAADAQABAAABAQDuwESFdAe14hVS6omeyX7edc...JQdf\[dq] 15059 } 15060 \f[R] 15061 .fi 15062 .PP 15063 And as an example return this on STDOUT 15064 .IP 15065 .nf 15066 \f[C] 15067 { 15068 \[dq]type\[dq]: \[dq]sftp\[dq], 15069 \[dq]_root\[dq]: \[dq]\[dq], 15070 \[dq]_obscure\[dq]: \[dq]pass\[dq], 15071 \[dq]user\[dq]: \[dq]me\[dq], 15072 \[dq]pass\[dq]: \[dq]mypassword\[dq], 15073 \[dq]host\[dq]: \[dq]sftp.example.com\[dq] 15074 } 15075 \f[R] 15076 .fi 15077 .PP 15078 This would mean that an SFTP backend would be created on the fly for the 15079 \f[C]user\f[R] and \f[C]pass\f[R]/\f[C]public_key\f[R] returned in the 15080 output to the host given. 15081 Note that since \f[C]_obscure\f[R] is set to \f[C]pass\f[R], rclone will 15082 obscure the \f[C]pass\f[R] parameter before creating the backend (which 15083 is required for sftp backends). 15084 .PP 15085 The program can manipulate the supplied \f[C]user\f[R] in any way, for 15086 example to make proxy to many different sftp backends, you could make 15087 the \f[C]user\f[R] be \f[C]user\[at]example.com\f[R] and then set the 15088 \f[C]host\f[R] to \f[C]example.com\f[R] in the output and the user to 15089 \f[C]user\f[R]. 15090 For security you\[aq]d probably want to restrict the \f[C]host\f[R] to a 15091 limited list. 15092 .PP 15093 Note that an internal cache is keyed on \f[C]user\f[R] so only use that 15094 for configuration, don\[aq]t use \f[C]pass\f[R] or \f[C]public_key\f[R]. 15095 This also means that if a user\[aq]s password or public-key is changed 15096 the cache will need to expire (which takes 5 mins) before it takes 15097 effect. 15098 .PP 15099 This can be used to build general purpose proxies to any kind of backend 15100 that rclone supports. 15101 .IP 15102 .nf 15103 \f[C] 15104 rclone serve webdav remote:path [flags] 15105 \f[R] 15106 .fi 15107 .SS Options 15108 .IP 15109 .nf 15110 \f[C] 15111 --addr stringArray IPaddress:Port or :Port to bind server to (default [127.0.0.1:8080]) 15112 --allow-origin string Origin which cross-domain request (CORS) can be executed from 15113 --auth-proxy string A program to use to create the backend from the auth 15114 --baseurl string Prefix for URLs - leave blank for root 15115 --cert string TLS PEM key (concatenation of certificate and CA certificate) 15116 --client-ca string Client certificate authority to verify clients with 15117 --dir-cache-time Duration Time to cache directory entries for (default 5m0s) 15118 --dir-perms FileMode Directory permissions (default 0777) 15119 --disable-dir-list Disable HTML directory list on GET request for a directory 15120 --etag-hash string Which hash to use for the ETag, or auto or blank for off 15121 --file-perms FileMode File permissions (default 0666) 15122 --gid uint32 Override the gid field set by the filesystem (not supported on Windows) (default 1000) 15123 -h, --help help for webdav 15124 --htpasswd string A htpasswd file - if not provided no authentication is done 15125 --key string TLS PEM Private key 15126 --max-header-bytes int Maximum size of request header (default 4096) 15127 --min-tls-version string Minimum TLS version that is acceptable (default \[dq]tls1.0\[dq]) 15128 --no-checksum Don\[aq]t compare checksums on up/download 15129 --no-modtime Don\[aq]t read/write the modification time (can speed things up) 15130 --no-seek Don\[aq]t allow seeking in files 15131 --pass string Password for authentication 15132 --poll-interval Duration Time to wait between polling for changes, must be smaller than dir-cache-time and only on supported remotes (set 0 to disable) (default 1m0s) 15133 --read-only Only allow read-only access 15134 --realm string Realm for authentication 15135 --salt string Password hashing salt (default \[dq]dlPL2MqE\[dq]) 15136 --server-read-timeout Duration Timeout for server reading data (default 1h0m0s) 15137 --server-write-timeout Duration Timeout for server writing data (default 1h0m0s) 15138 --template string User-specified template 15139 --uid uint32 Override the uid field set by the filesystem (not supported on Windows) (default 1000) 15140 --umask int Override the permission bits set by the filesystem (not supported on Windows) (default 2) 15141 --user string User name for authentication 15142 --vfs-block-norm-dupes If duplicate filenames exist in the same directory (after normalization), log an error and hide the duplicates (may have a performance cost) 15143 --vfs-cache-max-age Duration Max time since last access of objects in the cache (default 1h0m0s) 15144 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache (default off) 15145 --vfs-cache-min-free-space SizeSuffix Target minimum free space on the disk containing the cache (default off) 15146 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off) 15147 --vfs-cache-poll-interval Duration Interval to poll the cache for stale objects (default 1m0s) 15148 --vfs-case-insensitive If a file name not found, find a case insensitive match 15149 --vfs-disk-space-total-size SizeSuffix Specify the total space of disk (default off) 15150 --vfs-fast-fingerprint Use fast (less accurate) fingerprints for change detection 15151 --vfs-read-ahead SizeSuffix Extra read ahead over --buffer-size when using cache-mode full 15152 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks (default 128Mi) 15153 --vfs-read-chunk-size-limit SizeSuffix If greater than --vfs-read-chunk-size, double the chunk size after each chunk read, until the limit is reached (\[aq]off\[aq] is unlimited) (default off) 15154 --vfs-read-wait Duration Time to wait for in-sequence read before seeking (default 20ms) 15155 --vfs-refresh Refreshes the directory cache recursively in the background on start 15156 --vfs-used-is-size rclone size Use the rclone size algorithm for Used size 15157 --vfs-write-back Duration Time to writeback files after last use when using cache (default 5s) 15158 --vfs-write-wait Duration Time to wait for in-sequence write before giving error (default 1s) 15159 \f[R] 15160 .fi 15161 .SS Filter Options 15162 .PP 15163 Flags for filtering directory listings. 15164 .IP 15165 .nf 15166 \f[C] 15167 --delete-excluded Delete files on dest excluded from sync 15168 --exclude stringArray Exclude files matching pattern 15169 --exclude-from stringArray Read file exclude patterns from file (use - to read from stdin) 15170 --exclude-if-present stringArray Exclude directories if filename is present 15171 --files-from stringArray Read list of source-file names from file (use - to read from stdin) 15172 --files-from-raw stringArray Read list of source-file names from file without any processing of lines (use - to read from stdin) 15173 -f, --filter stringArray Add a file filtering rule 15174 --filter-from stringArray Read file filtering patterns from a file (use - to read from stdin) 15175 --ignore-case Ignore case in filters (case insensitive) 15176 --include stringArray Include files matching pattern 15177 --include-from stringArray Read file include patterns from file (use - to read from stdin) 15178 --max-age Duration Only transfer files younger than this in s or suffix ms|s|m|h|d|w|M|y (default off) 15179 --max-depth int If set limits the recursion depth to this (default -1) 15180 --max-size SizeSuffix Only transfer files smaller than this in KiB or suffix B|K|M|G|T|P (default off) 15181 --metadata-exclude stringArray Exclude metadatas matching pattern 15182 --metadata-exclude-from stringArray Read metadata exclude patterns from file (use - to read from stdin) 15183 --metadata-filter stringArray Add a metadata filtering rule 15184 --metadata-filter-from stringArray Read metadata filtering patterns from a file (use - to read from stdin) 15185 --metadata-include stringArray Include metadatas matching pattern 15186 --metadata-include-from stringArray Read metadata include patterns from file (use - to read from stdin) 15187 --min-age Duration Only transfer files older than this in s or suffix ms|s|m|h|d|w|M|y (default off) 15188 --min-size SizeSuffix Only transfer files bigger than this in KiB or suffix B|K|M|G|T|P (default off) 15189 \f[R] 15190 .fi 15191 .PP 15192 See the global flags page (https://rclone.org/flags/) for global options 15193 not listed here. 15194 .SH SEE ALSO 15195 .IP \[bu] 2 15196 rclone serve (https://rclone.org/commands/rclone_serve/) - Serve a 15197 remote over a protocol. 15198 .SH rclone settier 15199 .PP 15200 Changes storage class/tier of objects in remote. 15201 .SS Synopsis 15202 .PP 15203 rclone settier changes storage tier or class at remote if supported. 15204 Few cloud storage services provides different storage classes on 15205 objects, for example AWS S3 and Glacier, Azure Blob storage - Hot, Cool 15206 and Archive, Google Cloud Storage, Regional Storage, Nearline, Coldline 15207 etc. 15208 .PP 15209 Note that, certain tier changes make objects not available to access 15210 immediately. 15211 For example tiering to archive in azure blob storage makes objects in 15212 frozen state, user can restore by setting tier to Hot/Cool, similarly S3 15213 to Glacier makes object inaccessible.true 15214 .PP 15215 You can use it to tier single object 15216 .IP 15217 .nf 15218 \f[C] 15219 rclone settier Cool remote:path/file 15220 \f[R] 15221 .fi 15222 .PP 15223 Or use rclone filters to set tier on only specific files 15224 .IP 15225 .nf 15226 \f[C] 15227 rclone --include \[dq]*.txt\[dq] settier Hot remote:path/dir 15228 \f[R] 15229 .fi 15230 .PP 15231 Or just provide remote directory and all files in directory will be 15232 tiered 15233 .IP 15234 .nf 15235 \f[C] 15236 rclone settier tier remote:path/dir 15237 \f[R] 15238 .fi 15239 .IP 15240 .nf 15241 \f[C] 15242 rclone settier tier remote:path [flags] 15243 \f[R] 15244 .fi 15245 .SS Options 15246 .IP 15247 .nf 15248 \f[C] 15249 -h, --help help for settier 15250 \f[R] 15251 .fi 15252 .PP 15253 See the global flags page (https://rclone.org/flags/) for global options 15254 not listed here. 15255 .SH SEE ALSO 15256 .IP \[bu] 2 15257 rclone (https://rclone.org/commands/rclone/) - Show help for rclone 15258 commands, flags and backends. 15259 .SH rclone test 15260 .PP 15261 Run a test command 15262 .SS Synopsis 15263 .PP 15264 Rclone test is used to run test commands. 15265 .PP 15266 Select which test command you want with the subcommand, eg 15267 .IP 15268 .nf 15269 \f[C] 15270 rclone test memory remote: 15271 \f[R] 15272 .fi 15273 .PP 15274 Each subcommand has its own options which you can see in their help. 15275 .PP 15276 \f[B]NB\f[R] Be careful running these commands, they may do strange 15277 things so reading their documentation first is recommended. 15278 .SS Options 15279 .IP 15280 .nf 15281 \f[C] 15282 -h, --help help for test 15283 \f[R] 15284 .fi 15285 .PP 15286 See the global flags page (https://rclone.org/flags/) for global options 15287 not listed here. 15288 .SH SEE ALSO 15289 .IP \[bu] 2 15290 rclone (https://rclone.org/commands/rclone/) - Show help for rclone 15291 commands, flags and backends. 15292 .IP \[bu] 2 15293 rclone test 15294 changenotify (https://rclone.org/commands/rclone_test_changenotify/) - 15295 Log any change notify requests for the remote passed in. 15296 .IP \[bu] 2 15297 rclone test 15298 histogram (https://rclone.org/commands/rclone_test_histogram/) - Makes a 15299 histogram of file name characters. 15300 .IP \[bu] 2 15301 rclone test info (https://rclone.org/commands/rclone_test_info/) - 15302 Discovers file name or other limitations for paths. 15303 .IP \[bu] 2 15304 rclone test makefile (https://rclone.org/commands/rclone_test_makefile/) 15305 - Make files with random contents of the size given 15306 .IP \[bu] 2 15307 rclone test 15308 makefiles (https://rclone.org/commands/rclone_test_makefiles/) - Make a 15309 random file hierarchy in a directory 15310 .IP \[bu] 2 15311 rclone test memory (https://rclone.org/commands/rclone_test_memory/) - 15312 Load all the objects at remote:path into memory and report memory stats. 15313 .SH rclone test changenotify 15314 .PP 15315 Log any change notify requests for the remote passed in. 15316 .IP 15317 .nf 15318 \f[C] 15319 rclone test changenotify remote: [flags] 15320 \f[R] 15321 .fi 15322 .SS Options 15323 .IP 15324 .nf 15325 \f[C] 15326 -h, --help help for changenotify 15327 --poll-interval Duration Time to wait between polling for changes (default 10s) 15328 \f[R] 15329 .fi 15330 .PP 15331 See the global flags page (https://rclone.org/flags/) for global options 15332 not listed here. 15333 .SH SEE ALSO 15334 .IP \[bu] 2 15335 rclone test (https://rclone.org/commands/rclone_test/) - Run a test 15336 command 15337 .SH rclone test histogram 15338 .PP 15339 Makes a histogram of file name characters. 15340 .SS Synopsis 15341 .PP 15342 This command outputs JSON which shows the histogram of characters used 15343 in filenames in the remote:path specified. 15344 .PP 15345 The data doesn\[aq]t contain any identifying information but is useful 15346 for the rclone developers when developing filename compression. 15347 .IP 15348 .nf 15349 \f[C] 15350 rclone test histogram [remote:path] [flags] 15351 \f[R] 15352 .fi 15353 .SS Options 15354 .IP 15355 .nf 15356 \f[C] 15357 -h, --help help for histogram 15358 \f[R] 15359 .fi 15360 .PP 15361 See the global flags page (https://rclone.org/flags/) for global options 15362 not listed here. 15363 .SH SEE ALSO 15364 .IP \[bu] 2 15365 rclone test (https://rclone.org/commands/rclone_test/) - Run a test 15366 command 15367 .SH rclone test info 15368 .PP 15369 Discovers file name or other limitations for paths. 15370 .SS Synopsis 15371 .PP 15372 rclone info discovers what filenames and upload methods are possible to 15373 write to the paths passed in and how long they can be. 15374 It can take some time. 15375 It will write test files into the remote:path passed in. 15376 It outputs a bit of go code for each one. 15377 .PP 15378 \f[B]NB\f[R] this can create undeletable files and other hazards - use 15379 with care 15380 .IP 15381 .nf 15382 \f[C] 15383 rclone test info [remote:path]+ [flags] 15384 \f[R] 15385 .fi 15386 .SS Options 15387 .IP 15388 .nf 15389 \f[C] 15390 --all Run all tests 15391 --check-base32768 Check can store all possible base32768 characters 15392 --check-control Check control characters 15393 --check-length Check max filename length 15394 --check-normalization Check UTF-8 Normalization 15395 --check-streaming Check uploads with indeterminate file size 15396 -h, --help help for info 15397 --upload-wait Duration Wait after writing a file (default 0s) 15398 --write-json string Write results to file 15399 \f[R] 15400 .fi 15401 .PP 15402 See the global flags page (https://rclone.org/flags/) for global options 15403 not listed here. 15404 .SH SEE ALSO 15405 .IP \[bu] 2 15406 rclone test (https://rclone.org/commands/rclone_test/) - Run a test 15407 command 15408 .SH rclone test makefile 15409 .PP 15410 Make files with random contents of the size given 15411 .IP 15412 .nf 15413 \f[C] 15414 rclone test makefile <size> [<file>]+ [flags] 15415 \f[R] 15416 .fi 15417 .SS Options 15418 .IP 15419 .nf 15420 \f[C] 15421 --ascii Fill files with random ASCII printable bytes only 15422 --chargen Fill files with a ASCII chargen pattern 15423 -h, --help help for makefile 15424 --pattern Fill files with a periodic pattern 15425 --seed int Seed for the random number generator (0 for random) (default 1) 15426 --sparse Make the files sparse (appear to be filled with ASCII 0x00) 15427 --zero Fill files with ASCII 0x00 15428 \f[R] 15429 .fi 15430 .PP 15431 See the global flags page (https://rclone.org/flags/) for global options 15432 not listed here. 15433 .SH SEE ALSO 15434 .IP \[bu] 2 15435 rclone test (https://rclone.org/commands/rclone_test/) - Run a test 15436 command 15437 .SH rclone test makefiles 15438 .PP 15439 Make a random file hierarchy in a directory 15440 .IP 15441 .nf 15442 \f[C] 15443 rclone test makefiles <dir> [flags] 15444 \f[R] 15445 .fi 15446 .SS Options 15447 .IP 15448 .nf 15449 \f[C] 15450 --ascii Fill files with random ASCII printable bytes only 15451 --chargen Fill files with a ASCII chargen pattern 15452 --files int Number of files to create (default 1000) 15453 --files-per-directory int Average number of files per directory (default 10) 15454 -h, --help help for makefiles 15455 --max-depth int Maximum depth of directory hierarchy (default 10) 15456 --max-file-size SizeSuffix Maximum size of files to create (default 100) 15457 --max-name-length int Maximum size of file names (default 12) 15458 --min-file-size SizeSuffix Minimum size of file to create 15459 --min-name-length int Minimum size of file names (default 4) 15460 --pattern Fill files with a periodic pattern 15461 --seed int Seed for the random number generator (0 for random) (default 1) 15462 --sparse Make the files sparse (appear to be filled with ASCII 0x00) 15463 --zero Fill files with ASCII 0x00 15464 \f[R] 15465 .fi 15466 .PP 15467 See the global flags page (https://rclone.org/flags/) for global options 15468 not listed here. 15469 .SH SEE ALSO 15470 .IP \[bu] 2 15471 rclone test (https://rclone.org/commands/rclone_test/) - Run a test 15472 command 15473 .SH rclone test memory 15474 .PP 15475 Load all the objects at remote:path into memory and report memory stats. 15476 .IP 15477 .nf 15478 \f[C] 15479 rclone test memory remote:path [flags] 15480 \f[R] 15481 .fi 15482 .SS Options 15483 .IP 15484 .nf 15485 \f[C] 15486 -h, --help help for memory 15487 \f[R] 15488 .fi 15489 .PP 15490 See the global flags page (https://rclone.org/flags/) for global options 15491 not listed here. 15492 .SH SEE ALSO 15493 .IP \[bu] 2 15494 rclone test (https://rclone.org/commands/rclone_test/) - Run a test 15495 command 15496 .SH rclone touch 15497 .PP 15498 Create new file or change file modification time. 15499 .SS Synopsis 15500 .PP 15501 Set the modification time on file(s) as specified by remote:path to have 15502 the current time. 15503 .PP 15504 If remote:path does not exist then a zero sized file will be created, 15505 unless \f[C]--no-create\f[R] or \f[C]--recursive\f[R] is provided. 15506 .PP 15507 If \f[C]--recursive\f[R] is used then recursively sets the modification 15508 time on all existing files that is found under the path. 15509 Filters are supported, and you can test with the \f[C]--dry-run\f[R] or 15510 the \f[C]--interactive\f[R]/\f[C]-i\f[R] flag. 15511 .PP 15512 If \f[C]--timestamp\f[R] is used then sets the modification time to that 15513 time instead of the current time. 15514 Times may be specified as one of: 15515 .IP \[bu] 2 15516 \[aq]YYMMDD\[aq] - e.g. 15517 17.10.30 15518 .IP \[bu] 2 15519 \[aq]YYYY-MM-DDTHH:MM:SS\[aq] - e.g. 15520 2006-01-02T15:04:05 15521 .IP \[bu] 2 15522 \[aq]YYYY-MM-DDTHH:MM:SS.SSS\[aq] - e.g. 15523 2006-01-02T15:04:05.123456789 15524 .PP 15525 Note that value of \f[C]--timestamp\f[R] is in UTC. 15526 If you want local time then add the \f[C]--localtime\f[R] flag. 15527 .IP 15528 .nf 15529 \f[C] 15530 rclone touch remote:path [flags] 15531 \f[R] 15532 .fi 15533 .SS Options 15534 .IP 15535 .nf 15536 \f[C] 15537 -h, --help help for touch 15538 --localtime Use localtime for timestamp, not UTC 15539 -C, --no-create Do not create the file if it does not exist (implied with --recursive) 15540 -R, --recursive Recursively touch all files 15541 -t, --timestamp string Use specified time instead of the current time of day 15542 \f[R] 15543 .fi 15544 .SS Important Options 15545 .PP 15546 Important flags useful for most commands. 15547 .IP 15548 .nf 15549 \f[C] 15550 -n, --dry-run Do a trial run with no permanent changes 15551 -i, --interactive Enable interactive mode 15552 -v, --verbose count Print lots more stuff (repeat for more) 15553 \f[R] 15554 .fi 15555 .SS Filter Options 15556 .PP 15557 Flags for filtering directory listings. 15558 .IP 15559 .nf 15560 \f[C] 15561 --delete-excluded Delete files on dest excluded from sync 15562 --exclude stringArray Exclude files matching pattern 15563 --exclude-from stringArray Read file exclude patterns from file (use - to read from stdin) 15564 --exclude-if-present stringArray Exclude directories if filename is present 15565 --files-from stringArray Read list of source-file names from file (use - to read from stdin) 15566 --files-from-raw stringArray Read list of source-file names from file without any processing of lines (use - to read from stdin) 15567 -f, --filter stringArray Add a file filtering rule 15568 --filter-from stringArray Read file filtering patterns from a file (use - to read from stdin) 15569 --ignore-case Ignore case in filters (case insensitive) 15570 --include stringArray Include files matching pattern 15571 --include-from stringArray Read file include patterns from file (use - to read from stdin) 15572 --max-age Duration Only transfer files younger than this in s or suffix ms|s|m|h|d|w|M|y (default off) 15573 --max-depth int If set limits the recursion depth to this (default -1) 15574 --max-size SizeSuffix Only transfer files smaller than this in KiB or suffix B|K|M|G|T|P (default off) 15575 --metadata-exclude stringArray Exclude metadatas matching pattern 15576 --metadata-exclude-from stringArray Read metadata exclude patterns from file (use - to read from stdin) 15577 --metadata-filter stringArray Add a metadata filtering rule 15578 --metadata-filter-from stringArray Read metadata filtering patterns from a file (use - to read from stdin) 15579 --metadata-include stringArray Include metadatas matching pattern 15580 --metadata-include-from stringArray Read metadata include patterns from file (use - to read from stdin) 15581 --min-age Duration Only transfer files older than this in s or suffix ms|s|m|h|d|w|M|y (default off) 15582 --min-size SizeSuffix Only transfer files bigger than this in KiB or suffix B|K|M|G|T|P (default off) 15583 \f[R] 15584 .fi 15585 .SS Listing Options 15586 .PP 15587 Flags for listing directories. 15588 .IP 15589 .nf 15590 \f[C] 15591 --default-time Time Time to show if modtime is unknown for files and directories (default 2000-01-01T00:00:00Z) 15592 --fast-list Use recursive list if available; uses more memory but fewer transactions 15593 \f[R] 15594 .fi 15595 .PP 15596 See the global flags page (https://rclone.org/flags/) for global options 15597 not listed here. 15598 .SH SEE ALSO 15599 .IP \[bu] 2 15600 rclone (https://rclone.org/commands/rclone/) - Show help for rclone 15601 commands, flags and backends. 15602 .SH rclone tree 15603 .PP 15604 List the contents of the remote in a tree like fashion. 15605 .SS Synopsis 15606 .PP 15607 rclone tree lists the contents of a remote in a similar way to the unix 15608 tree command. 15609 .PP 15610 For example 15611 .IP 15612 .nf 15613 \f[C] 15614 $ rclone tree remote:path 15615 / 15616 \[u251C]\[u2500]\[u2500] file1 15617 \[u251C]\[u2500]\[u2500] file2 15618 \[u251C]\[u2500]\[u2500] file3 15619 \[u2514]\[u2500]\[u2500] subdir 15620 \[u251C]\[u2500]\[u2500] file4 15621 \[u2514]\[u2500]\[u2500] file5 15622 15623 1 directories, 5 files 15624 \f[R] 15625 .fi 15626 .PP 15627 You can use any of the filtering options with the tree command (e.g. 15628 \f[C]--include\f[R] and \f[C]--exclude\f[R]. 15629 You can also use \f[C]--fast-list\f[R]. 15630 .PP 15631 The tree command has many options for controlling the listing which are 15632 compatible with the tree command, for example you can include file sizes 15633 with \f[C]--size\f[R]. 15634 Note that not all of them have short options as they conflict with 15635 rclone\[aq]s short options. 15636 .PP 15637 For a more interactive navigation of the remote see the 15638 ncdu (https://rclone.org/commands/rclone_ncdu/) command. 15639 .IP 15640 .nf 15641 \f[C] 15642 rclone tree remote:path [flags] 15643 \f[R] 15644 .fi 15645 .SS Options 15646 .IP 15647 .nf 15648 \f[C] 15649 -a, --all All files are listed (list . files too) 15650 -d, --dirs-only List directories only 15651 --dirsfirst List directories before files (-U disables) 15652 --full-path Print the full path prefix for each file 15653 -h, --help help for tree 15654 --level int Descend only level directories deep 15655 -D, --modtime Print the date of last modification. 15656 --noindent Don\[aq]t print indentation lines 15657 --noreport Turn off file/directory count at end of tree listing 15658 -o, --output string Output to file instead of stdout 15659 -p, --protections Print the protections for each file. 15660 -Q, --quote Quote filenames with double quotes. 15661 -s, --size Print the size in bytes of each file. 15662 --sort string Select sort: name,version,size,mtime,ctime 15663 --sort-ctime Sort files by last status change time 15664 -t, --sort-modtime Sort files by last modification time 15665 -r, --sort-reverse Reverse the order of the sort 15666 -U, --unsorted Leave files unsorted 15667 --version Sort files alphanumerically by version 15668 \f[R] 15669 .fi 15670 .SS Filter Options 15671 .PP 15672 Flags for filtering directory listings. 15673 .IP 15674 .nf 15675 \f[C] 15676 --delete-excluded Delete files on dest excluded from sync 15677 --exclude stringArray Exclude files matching pattern 15678 --exclude-from stringArray Read file exclude patterns from file (use - to read from stdin) 15679 --exclude-if-present stringArray Exclude directories if filename is present 15680 --files-from stringArray Read list of source-file names from file (use - to read from stdin) 15681 --files-from-raw stringArray Read list of source-file names from file without any processing of lines (use - to read from stdin) 15682 -f, --filter stringArray Add a file filtering rule 15683 --filter-from stringArray Read file filtering patterns from a file (use - to read from stdin) 15684 --ignore-case Ignore case in filters (case insensitive) 15685 --include stringArray Include files matching pattern 15686 --include-from stringArray Read file include patterns from file (use - to read from stdin) 15687 --max-age Duration Only transfer files younger than this in s or suffix ms|s|m|h|d|w|M|y (default off) 15688 --max-depth int If set limits the recursion depth to this (default -1) 15689 --max-size SizeSuffix Only transfer files smaller than this in KiB or suffix B|K|M|G|T|P (default off) 15690 --metadata-exclude stringArray Exclude metadatas matching pattern 15691 --metadata-exclude-from stringArray Read metadata exclude patterns from file (use - to read from stdin) 15692 --metadata-filter stringArray Add a metadata filtering rule 15693 --metadata-filter-from stringArray Read metadata filtering patterns from a file (use - to read from stdin) 15694 --metadata-include stringArray Include metadatas matching pattern 15695 --metadata-include-from stringArray Read metadata include patterns from file (use - to read from stdin) 15696 --min-age Duration Only transfer files older than this in s or suffix ms|s|m|h|d|w|M|y (default off) 15697 --min-size SizeSuffix Only transfer files bigger than this in KiB or suffix B|K|M|G|T|P (default off) 15698 \f[R] 15699 .fi 15700 .SS Listing Options 15701 .PP 15702 Flags for listing directories. 15703 .IP 15704 .nf 15705 \f[C] 15706 --default-time Time Time to show if modtime is unknown for files and directories (default 2000-01-01T00:00:00Z) 15707 --fast-list Use recursive list if available; uses more memory but fewer transactions 15708 \f[R] 15709 .fi 15710 .PP 15711 See the global flags page (https://rclone.org/flags/) for global options 15712 not listed here. 15713 .SH SEE ALSO 15714 .IP \[bu] 2 15715 rclone (https://rclone.org/commands/rclone/) - Show help for rclone 15716 commands, flags and backends. 15717 .SS Copying single files 15718 .PP 15719 rclone normally syncs or copies directories. 15720 However, if the source remote points to a file, rclone will just copy 15721 that file. 15722 The destination remote must point to a directory - rclone will give the 15723 error 15724 \f[C]Failed to create file system for \[dq]remote:file\[dq]: is a file not a directory\f[R] 15725 if it isn\[aq]t. 15726 .PP 15727 For example, suppose you have a remote with a file in called 15728 \f[C]test.jpg\f[R], then you could copy just that file like this 15729 .IP 15730 .nf 15731 \f[C] 15732 rclone copy remote:test.jpg /tmp/download 15733 \f[R] 15734 .fi 15735 .PP 15736 The file \f[C]test.jpg\f[R] will be placed inside 15737 \f[C]/tmp/download\f[R]. 15738 .PP 15739 This is equivalent to specifying 15740 .IP 15741 .nf 15742 \f[C] 15743 rclone copy --files-from /tmp/files remote: /tmp/download 15744 \f[R] 15745 .fi 15746 .PP 15747 Where \f[C]/tmp/files\f[R] contains the single line 15748 .IP 15749 .nf 15750 \f[C] 15751 test.jpg 15752 \f[R] 15753 .fi 15754 .PP 15755 It is recommended to use \f[C]copy\f[R] when copying individual files, 15756 not \f[C]sync\f[R]. 15757 They have pretty much the same effect but \f[C]copy\f[R] will use a lot 15758 less memory. 15759 .SS Syntax of remote paths 15760 .PP 15761 The syntax of the paths passed to the rclone command are as follows. 15762 .SS /path/to/dir 15763 .PP 15764 This refers to the local file system. 15765 .PP 15766 On Windows \f[C]\[rs]\f[R] may be used instead of \f[C]/\f[R] in local 15767 paths \f[B]only\f[R], non local paths must use \f[C]/\f[R]. 15768 See local filesystem (https://rclone.org/local/#paths-on-windows) 15769 documentation for more about Windows-specific paths. 15770 .PP 15771 These paths needn\[aq]t start with a leading \f[C]/\f[R] - if they 15772 don\[aq]t then they will be relative to the current directory. 15773 .SS remote:path/to/dir 15774 .PP 15775 This refers to a directory \f[C]path/to/dir\f[R] on \f[C]remote:\f[R] as 15776 defined in the config file (configured with \f[C]rclone config\f[R]). 15777 .SS remote:/path/to/dir 15778 .PP 15779 On most backends this is refers to the same directory as 15780 \f[C]remote:path/to/dir\f[R] and that format should be preferred. 15781 On a very small number of remotes (FTP, SFTP, Dropbox for business) this 15782 will refer to a different directory. 15783 On these, paths without a leading \f[C]/\f[R] will refer to your 15784 \[dq]home\[dq] directory and paths with a leading \f[C]/\f[R] will refer 15785 to the root. 15786 .SS :backend:path/to/dir 15787 .PP 15788 This is an advanced form for creating remotes on the fly. 15789 \f[C]backend\f[R] should be the name or prefix of a backend (the 15790 \f[C]type\f[R] in the config file) and all the configuration for the 15791 backend should be provided on the command line (or in environment 15792 variables). 15793 .PP 15794 Here are some examples: 15795 .IP 15796 .nf 15797 \f[C] 15798 rclone lsd --http-url https://pub.rclone.org :http: 15799 \f[R] 15800 .fi 15801 .PP 15802 To list all the directories in the root of 15803 \f[C]https://pub.rclone.org/\f[R]. 15804 .IP 15805 .nf 15806 \f[C] 15807 rclone lsf --http-url https://example.com :http:path/to/dir 15808 \f[R] 15809 .fi 15810 .PP 15811 To list files and directories in 15812 \f[C]https://example.com/path/to/dir/\f[R] 15813 .IP 15814 .nf 15815 \f[C] 15816 rclone copy --http-url https://example.com :http:path/to/dir /tmp/dir 15817 \f[R] 15818 .fi 15819 .PP 15820 To copy files and directories in 15821 \f[C]https://example.com/path/to/dir\f[R] to \f[C]/tmp/dir\f[R]. 15822 .IP 15823 .nf 15824 \f[C] 15825 rclone copy --sftp-host example.com :sftp:path/to/dir /tmp/dir 15826 \f[R] 15827 .fi 15828 .PP 15829 To copy files and directories from \f[C]example.com\f[R] in the relative 15830 directory \f[C]path/to/dir\f[R] to \f[C]/tmp/dir\f[R] using sftp. 15831 .SS Connection strings 15832 .PP 15833 The above examples can also be written using a connection string syntax, 15834 so instead of providing the arguments as command line parameters 15835 \f[C]--http-url https://pub.rclone.org\f[R] they are provided as part of 15836 the remote specification as a kind of connection string. 15837 .IP 15838 .nf 15839 \f[C] 15840 rclone lsd \[dq]:http,url=\[aq]https://pub.rclone.org\[aq]:\[dq] 15841 rclone lsf \[dq]:http,url=\[aq]https://example.com\[aq]:path/to/dir\[dq] 15842 rclone copy \[dq]:http,url=\[aq]https://example.com\[aq]:path/to/dir\[dq] /tmp/dir 15843 rclone copy :sftp,host=example.com:path/to/dir /tmp/dir 15844 \f[R] 15845 .fi 15846 .PP 15847 These can apply to modify existing remotes as well as create new remotes 15848 with the on the fly syntax. 15849 This example is equivalent to adding the 15850 \f[C]--drive-shared-with-me\f[R] parameter to the remote 15851 \f[C]gdrive:\f[R]. 15852 .IP 15853 .nf 15854 \f[C] 15855 rclone lsf \[dq]gdrive,shared_with_me:path/to/dir\[dq] 15856 \f[R] 15857 .fi 15858 .PP 15859 The major advantage to using the connection string style syntax is that 15860 it only applies to the remote, not to all the remotes of that type of 15861 the command line. 15862 A common confusion is this attempt to copy a file shared on google drive 15863 to the normal drive which \f[B]does not work\f[R] because the 15864 \f[C]--drive-shared-with-me\f[R] flag applies to both the source and the 15865 destination. 15866 .IP 15867 .nf 15868 \f[C] 15869 rclone copy --drive-shared-with-me gdrive:shared-file.txt gdrive: 15870 \f[R] 15871 .fi 15872 .PP 15873 However using the connection string syntax, this does work. 15874 .IP 15875 .nf 15876 \f[C] 15877 rclone copy \[dq]gdrive,shared_with_me:shared-file.txt\[dq] gdrive: 15878 \f[R] 15879 .fi 15880 .PP 15881 Note that the connection string only affects the options of the 15882 immediate backend. 15883 If for example gdriveCrypt is a crypt based on gdrive, then the 15884 following command \f[B]will not work\f[R] as intended, because 15885 \f[C]shared_with_me\f[R] is ignored by the crypt backend: 15886 .IP 15887 .nf 15888 \f[C] 15889 rclone copy \[dq]gdriveCrypt,shared_with_me:shared-file.txt\[dq] gdriveCrypt: 15890 \f[R] 15891 .fi 15892 .PP 15893 The connection strings have the following syntax 15894 .IP 15895 .nf 15896 \f[C] 15897 remote,parameter=value,parameter2=value2:path/to/dir 15898 :backend,parameter=value,parameter2=value2:path/to/dir 15899 \f[R] 15900 .fi 15901 .PP 15902 If the \f[C]parameter\f[R] has a \f[C]:\f[R] or \f[C],\f[R] then it must 15903 be placed in quotes \f[C]\[dq]\f[R] or \f[C]\[aq]\f[R], so 15904 .IP 15905 .nf 15906 \f[C] 15907 remote,parameter=\[dq]colon:value\[dq],parameter2=\[dq]comma,value\[dq]:path/to/dir 15908 :backend,parameter=\[aq]colon:value\[aq],parameter2=\[aq]comma,value\[aq]:path/to/dir 15909 \f[R] 15910 .fi 15911 .PP 15912 If a quoted value needs to include that quote, then it should be 15913 doubled, so 15914 .IP 15915 .nf 15916 \f[C] 15917 remote,parameter=\[dq]with\[dq]\[dq]quote\[dq],parameter2=\[aq]with\[aq]\[aq]quote\[aq]:path/to/dir 15918 \f[R] 15919 .fi 15920 .PP 15921 This will make \f[C]parameter\f[R] be \f[C]with\[dq]quote\f[R] and 15922 \f[C]parameter2\f[R] be \f[C]with\[aq]quote\f[R]. 15923 .PP 15924 If you leave off the \f[C]=parameter\f[R] then rclone will substitute 15925 \f[C]=true\f[R] which works very well with flags. 15926 For example, to use s3 configured in the environment you could use: 15927 .IP 15928 .nf 15929 \f[C] 15930 rclone lsd :s3,env_auth: 15931 \f[R] 15932 .fi 15933 .PP 15934 Which is equivalent to 15935 .IP 15936 .nf 15937 \f[C] 15938 rclone lsd :s3,env_auth=true: 15939 \f[R] 15940 .fi 15941 .PP 15942 Note that on the command line you might need to surround these 15943 connection strings with \f[C]\[dq]\f[R] or \f[C]\[aq]\f[R] to stop the 15944 shell interpreting any special characters within them. 15945 .PP 15946 If you are a shell master then you\[aq]ll know which strings are OK and 15947 which aren\[aq]t, but if you aren\[aq]t sure then enclose them in 15948 \f[C]\[dq]\f[R] and use \f[C]\[aq]\f[R] as the inside quote. 15949 This syntax works on all OSes. 15950 .IP 15951 .nf 15952 \f[C] 15953 rclone copy \[dq]:http,url=\[aq]https://example.com\[aq]:path/to/dir\[dq] /tmp/dir 15954 \f[R] 15955 .fi 15956 .PP 15957 On Linux/macOS some characters are still interpreted inside 15958 \f[C]\[dq]\f[R] strings in the shell (notably \f[C]\[rs]\f[R] and 15959 \f[C]$\f[R] and \f[C]\[dq]\f[R]) so if your strings contain those you 15960 can swap the roles of \f[C]\[dq]\f[R] and \f[C]\[aq]\f[R] thus. 15961 (This syntax does not work on Windows.) 15962 .IP 15963 .nf 15964 \f[C] 15965 rclone copy \[aq]:http,url=\[dq]https://example.com\[dq]:path/to/dir\[aq] /tmp/dir 15966 \f[R] 15967 .fi 15968 .SS Connection strings, config and logging 15969 .PP 15970 If you supply extra configuration to a backend by command line flag, 15971 environment variable or connection string then rclone will add a suffix 15972 based on the hash of the config to the name of the remote, eg 15973 .IP 15974 .nf 15975 \f[C] 15976 rclone -vv lsf --s3-chunk-size 20M s3: 15977 \f[R] 15978 .fi 15979 .PP 15980 Has the log message 15981 .IP 15982 .nf 15983 \f[C] 15984 DEBUG : s3: detected overridden config - adding \[dq]{Srj1p}\[dq] suffix to name 15985 \f[R] 15986 .fi 15987 .PP 15988 This is so rclone can tell the modified remote apart from the unmodified 15989 remote when caching the backends. 15990 .PP 15991 This should only be noticeable in the logs. 15992 .PP 15993 This means that on the fly backends such as 15994 .IP 15995 .nf 15996 \f[C] 15997 rclone -vv lsf :s3,env_auth: 15998 \f[R] 15999 .fi 16000 .PP 16001 Will get their own names 16002 .IP 16003 .nf 16004 \f[C] 16005 DEBUG : :s3: detected overridden config - adding \[dq]{YTu53}\[dq] suffix to name 16006 \f[R] 16007 .fi 16008 .SS Valid remote names 16009 .PP 16010 Remote names are case sensitive, and must adhere to the following rules: 16011 - May contain number, letter, \f[C]_\f[R], \f[C]-\f[R], \f[C].\f[R], 16012 \f[C]+\f[R], \f[C]\[at]\f[R] and space. 16013 - May not start with \f[C]-\f[R] or space. 16014 - May not end with space. 16015 .PP 16016 Starting with rclone version 1.61, any Unicode numbers and letters are 16017 allowed, while in older versions it was limited to plain ASCII (0-9, 16018 A-Z, a-z). 16019 If you use the same rclone configuration from different shells, which 16020 may be configured with different character encoding, you must be 16021 cautious to use characters that are possible to write in all of them. 16022 This is mostly a problem on Windows, where the console traditionally 16023 uses a non-Unicode character set - defined by the so-called \[dq]code 16024 page\[dq]. 16025 .PP 16026 Do not use single character names on Windows as it creates ambiguity 16027 with Windows drives\[aq] names, e.g.: remote called \f[C]C\f[R] is 16028 indistinguishable from \f[C]C\f[R] drive. 16029 Rclone will always assume that single letter name refers to a drive. 16030 .SS Quoting and the shell 16031 .PP 16032 When you are typing commands to your computer you are using something 16033 called the command line shell. 16034 This interprets various characters in an OS specific way. 16035 .PP 16036 Here are some gotchas which may help users unfamiliar with the shell 16037 rules 16038 .SS Linux / OSX 16039 .PP 16040 If your names have spaces or shell metacharacters (e.g. 16041 \f[C]*\f[R], \f[C]?\f[R], \f[C]$\f[R], \f[C]\[aq]\f[R], \f[C]\[dq]\f[R], 16042 etc.) then you must quote them. 16043 Use single quotes \f[C]\[aq]\f[R] by default. 16044 .IP 16045 .nf 16046 \f[C] 16047 rclone copy \[aq]Important files?\[aq] remote:backup 16048 \f[R] 16049 .fi 16050 .PP 16051 If you want to send a \f[C]\[aq]\f[R] you will need to use 16052 \f[C]\[dq]\f[R], e.g. 16053 .IP 16054 .nf 16055 \f[C] 16056 rclone copy \[dq]O\[aq]Reilly Reviews\[dq] remote:backup 16057 \f[R] 16058 .fi 16059 .PP 16060 The rules for quoting metacharacters are complicated and if you want the 16061 full details you\[aq]ll have to consult the manual page for your shell. 16062 .SS Windows 16063 .PP 16064 If your names have spaces in you need to put them in \f[C]\[dq]\f[R], 16065 e.g. 16066 .IP 16067 .nf 16068 \f[C] 16069 rclone copy \[dq]E:\[rs]folder name\[rs]folder name\[rs]folder name\[dq] remote:backup 16070 \f[R] 16071 .fi 16072 .PP 16073 If you are using the root directory on its own then don\[aq]t quote it 16074 (see #464 (https://github.com/artpar/artpar/issues/464) for why), e.g. 16075 .IP 16076 .nf 16077 \f[C] 16078 rclone copy E:\[rs] remote:backup 16079 \f[R] 16080 .fi 16081 .SS Copying files or directories with \f[C]:\f[R] in the names 16082 .PP 16083 rclone uses \f[C]:\f[R] to mark a remote name. 16084 This is, however, a valid filename component in non-Windows OSes. 16085 The remote name parser will only search for a \f[C]:\f[R] up to the 16086 first \f[C]/\f[R] so if you need to act on a file or directory like this 16087 then use the full path starting with a \f[C]/\f[R], or use \f[C]./\f[R] 16088 as a current directory prefix. 16089 .PP 16090 So to sync a directory called \f[C]sync:me\f[R] to a remote called 16091 \f[C]remote:\f[R] use 16092 .IP 16093 .nf 16094 \f[C] 16095 rclone sync --interactive ./sync:me remote:path 16096 \f[R] 16097 .fi 16098 .PP 16099 or 16100 .IP 16101 .nf 16102 \f[C] 16103 rclone sync --interactive /full/path/to/sync:me remote:path 16104 \f[R] 16105 .fi 16106 .SS Server Side Copy 16107 .PP 16108 Most remotes (but not all - see the 16109 overview (https://rclone.org/overview/#optional-features)) support 16110 server-side copy. 16111 .PP 16112 This means if you want to copy one folder to another then rclone 16113 won\[aq]t download all the files and re-upload them; it will instruct 16114 the server to copy them in place. 16115 .PP 16116 Eg 16117 .IP 16118 .nf 16119 \f[C] 16120 rclone copy s3:oldbucket s3:newbucket 16121 \f[R] 16122 .fi 16123 .PP 16124 Will copy the contents of \f[C]oldbucket\f[R] to \f[C]newbucket\f[R] 16125 without downloading and re-uploading. 16126 .PP 16127 Remotes which don\[aq]t support server-side copy \f[B]will\f[R] download 16128 and re-upload in this case. 16129 .PP 16130 Server side copies are used with \f[C]sync\f[R] and \f[C]copy\f[R] and 16131 will be identified in the log when using the \f[C]-v\f[R] flag. 16132 The \f[C]move\f[R] command may also use them if remote doesn\[aq]t 16133 support server-side move directly. 16134 This is done by issuing a server-side copy then a delete which is much 16135 quicker than a download and re-upload. 16136 .PP 16137 Server side copies will only be attempted if the remote names are the 16138 same. 16139 .PP 16140 This can be used when scripting to make aged backups efficiently, e.g. 16141 .IP 16142 .nf 16143 \f[C] 16144 rclone sync --interactive remote:current-backup remote:previous-backup 16145 rclone sync --interactive /path/to/files remote:current-backup 16146 \f[R] 16147 .fi 16148 .SS Metadata support 16149 .PP 16150 Metadata is data about a file (or directory) which isn\[aq]t the 16151 contents of the file (or directory). 16152 Normally rclone only preserves the modification time and the content 16153 (MIME) type where possible. 16154 .PP 16155 Rclone supports preserving all the available metadata on files and 16156 directories when using the \f[C]--metadata\f[R] or \f[C]-M\f[R] flag. 16157 .PP 16158 Exactly what metadata is supported and what that support means depends 16159 on the backend. 16160 Backends that support metadata have a metadata section in their docs and 16161 are listed in the features table (https://rclone.org/overview/#features) 16162 (Eg local (https://rclone.org/local/#metadata), s3) 16163 .PP 16164 Some backends don\[aq]t support metadata, some only support metadata on 16165 files and some support metadata on both files and directories. 16166 .PP 16167 Rclone only supports a one-time sync of metadata. 16168 This means that metadata will be synced from the source object to the 16169 destination object only when the source object has changed and needs to 16170 be re-uploaded. 16171 If the metadata subsequently changes on the source object without 16172 changing the object itself then it won\[aq]t be synced to the 16173 destination object. 16174 This is in line with the way rclone syncs \f[C]Content-Type\f[R] without 16175 the \f[C]--metadata\f[R] flag. 16176 .PP 16177 Using \f[C]--metadata\f[R] when syncing from local to local will 16178 preserve file attributes such as file mode, owner, extended attributes 16179 (not Windows). 16180 .PP 16181 Note that arbitrary metadata may be added to objects using the 16182 \f[C]--metadata-set key=value\f[R] flag when the object is first 16183 uploaded. 16184 This flag can be repeated as many times as necessary. 16185 .PP 16186 The --metadata-mapper flag can be used to pass the name of a program in 16187 which can transform metadata when it is being copied from source to 16188 destination. 16189 .PP 16190 Rclone supports \f[C]--metadata-set\f[R] and \f[C]--metadata-mapper\f[R] 16191 when doing sever side \f[C]Move\f[R] and server side \f[C]Copy\f[R], but 16192 not when doing server side \f[C]DirMove\f[R] (renaming a directory) as 16193 this would involve recursing into the directory. 16194 Note that you can disable \f[C]DirMove\f[R] with 16195 \f[C]--disable DirMove\f[R] and rclone will revert back to using 16196 \f[C]Move\f[R] for each individual object where \f[C]--metadata-set\f[R] 16197 and \f[C]--metadata-mapper\f[R] are supported. 16198 .SS Types of metadata 16199 .PP 16200 Metadata is divided into two type. 16201 System metadata and User metadata. 16202 .PP 16203 Metadata which the backend uses itself is called system metadata. 16204 For example on the local backend the system metadata \f[C]uid\f[R] will 16205 store the user ID of the file when used on a unix based platform. 16206 .PP 16207 Arbitrary metadata is called user metadata and this can be set however 16208 is desired. 16209 .PP 16210 When objects are copied from backend to backend, they will attempt to 16211 interpret system metadata if it is supplied. 16212 Metadata may change from being user metadata to system metadata as 16213 objects are copied between different backends. 16214 For example copying an object from s3 sets the \f[C]content-type\f[R] 16215 metadata. 16216 In a backend which understands this (like \f[C]azureblob\f[R]) this will 16217 become the Content-Type of the object. 16218 In a backend which doesn\[aq]t understand this (like the \f[C]local\f[R] 16219 backend) this will become user metadata. 16220 However should the local object be copied back to s3, the Content-Type 16221 will be set correctly. 16222 .SS Metadata framework 16223 .PP 16224 Rclone implements a metadata framework which can read metadata from an 16225 object and write it to the object when (and only when) it is being 16226 uploaded. 16227 .PP 16228 This metadata is stored as a dictionary with string keys and string 16229 values. 16230 .PP 16231 There are some limits on the names of the keys (these may be clarified 16232 further in the future). 16233 .IP \[bu] 2 16234 must be lower case 16235 .IP \[bu] 2 16236 may be \f[C]a-z\f[R] \f[C]0-9\f[R] containing \f[C].\f[R] \f[C]-\f[R] or 16237 \f[C]_\f[R] 16238 .IP \[bu] 2 16239 length is backend dependent 16240 .PP 16241 Each backend can provide system metadata that it understands. 16242 Some backends can also store arbitrary user metadata. 16243 .PP 16244 Where possible the key names are standardized, so, for example, it is 16245 possible to copy object metadata from s3 to azureblob for example and 16246 metadata will be translated appropriately. 16247 .PP 16248 Some backends have limits on the size of the metadata and rclone will 16249 give errors on upload if they are exceeded. 16250 .SS Metadata preservation 16251 .PP 16252 The goal of the implementation is to 16253 .IP "1." 3 16254 Preserve metadata if at all possible 16255 .IP "2." 3 16256 Interpret metadata if at all possible 16257 .PP 16258 The consequences of 1 is that you can copy an S3 object to a local disk 16259 then back to S3 losslessly. 16260 Likewise you can copy a local file with file attributes and xattrs from 16261 local disk to s3 and back again losslessly. 16262 .PP 16263 The consequence of 2 is that you can copy an S3 object with metadata to 16264 Azureblob (say) and have the metadata appear on the Azureblob object 16265 also. 16266 .SS Standard system metadata 16267 .PP 16268 Here is a table of standard system metadata which, if appropriate, a 16269 backend may implement. 16270 .PP 16271 .TS 16272 tab(@); 16273 lw(34.2n) lw(21.2n) lw(14.7n). 16274 T{ 16275 key 16276 T}@T{ 16277 description 16278 T}@T{ 16279 example 16280 T} 16281 _ 16282 T{ 16283 mode 16284 T}@T{ 16285 File type and mode: octal, unix style 16286 T}@T{ 16287 0100664 16288 T} 16289 T{ 16290 uid 16291 T}@T{ 16292 User ID of owner: decimal number 16293 T}@T{ 16294 500 16295 T} 16296 T{ 16297 gid 16298 T}@T{ 16299 Group ID of owner: decimal number 16300 T}@T{ 16301 500 16302 T} 16303 T{ 16304 rdev 16305 T}@T{ 16306 Device ID (if special file) => hexadecimal 16307 T}@T{ 16308 0 16309 T} 16310 T{ 16311 atime 16312 T}@T{ 16313 Time of last access: RFC 3339 16314 T}@T{ 16315 2006-01-02T15:04:05.999999999Z07:00 16316 T} 16317 T{ 16318 mtime 16319 T}@T{ 16320 Time of last modification: RFC 3339 16321 T}@T{ 16322 2006-01-02T15:04:05.999999999Z07:00 16323 T} 16324 T{ 16325 btime 16326 T}@T{ 16327 Time of file creation (birth): RFC 3339 16328 T}@T{ 16329 2006-01-02T15:04:05.999999999Z07:00 16330 T} 16331 T{ 16332 utime 16333 T}@T{ 16334 Time of file upload: RFC 3339 16335 T}@T{ 16336 2006-01-02T15:04:05.999999999Z07:00 16337 T} 16338 T{ 16339 cache-control 16340 T}@T{ 16341 Cache-Control header 16342 T}@T{ 16343 no-cache 16344 T} 16345 T{ 16346 content-disposition 16347 T}@T{ 16348 Content-Disposition header 16349 T}@T{ 16350 inline 16351 T} 16352 T{ 16353 content-encoding 16354 T}@T{ 16355 Content-Encoding header 16356 T}@T{ 16357 gzip 16358 T} 16359 T{ 16360 content-language 16361 T}@T{ 16362 Content-Language header 16363 T}@T{ 16364 en-US 16365 T} 16366 T{ 16367 content-type 16368 T}@T{ 16369 Content-Type header 16370 T}@T{ 16371 text/plain 16372 T} 16373 .TE 16374 .PP 16375 The metadata keys \f[C]mtime\f[R] and \f[C]content-type\f[R] will take 16376 precedence if supplied in the metadata over reading the 16377 \f[C]Content-Type\f[R] or modification time of the source object. 16378 .PP 16379 Hashes are not included in system metadata as there is a well defined 16380 way of reading those already. 16381 .SS Options 16382 .PP 16383 Rclone has a number of options to control its behaviour. 16384 .PP 16385 Options that take parameters can have the values passed in two ways, 16386 \f[C]--option=value\f[R] or \f[C]--option value\f[R]. 16387 However boolean (true/false) options behave slightly differently to the 16388 other options in that \f[C]--boolean\f[R] sets the option to 16389 \f[C]true\f[R] and the absence of the flag sets it to \f[C]false\f[R]. 16390 It is also possible to specify \f[C]--boolean=false\f[R] or 16391 \f[C]--boolean=true\f[R]. 16392 Note that \f[C]--boolean false\f[R] is not valid - this is parsed as 16393 \f[C]--boolean\f[R] and the \f[C]false\f[R] is parsed as an extra 16394 command line argument for rclone. 16395 .SS Time or duration options 16396 .PP 16397 TIME or DURATION options can be specified as a duration string or a time 16398 string. 16399 .PP 16400 A duration string is a possibly signed sequence of decimal numbers, each 16401 with optional fraction and a unit suffix, such as \[dq]300ms\[dq], 16402 \[dq]-1.5h\[dq] or \[dq]2h45m\[dq]. 16403 Default units are seconds or the following abbreviations are valid: 16404 .IP \[bu] 2 16405 \f[C]ms\f[R] - Milliseconds 16406 .IP \[bu] 2 16407 \f[C]s\f[R] - Seconds 16408 .IP \[bu] 2 16409 \f[C]m\f[R] - Minutes 16410 .IP \[bu] 2 16411 \f[C]h\f[R] - Hours 16412 .IP \[bu] 2 16413 \f[C]d\f[R] - Days 16414 .IP \[bu] 2 16415 \f[C]w\f[R] - Weeks 16416 .IP \[bu] 2 16417 \f[C]M\f[R] - Months 16418 .IP \[bu] 2 16419 \f[C]y\f[R] - Years 16420 .PP 16421 These can also be specified as an absolute time in the following 16422 formats: 16423 .IP \[bu] 2 16424 RFC3339 - e.g. 16425 \f[C]2006-01-02T15:04:05Z\f[R] or \f[C]2006-01-02T15:04:05+07:00\f[R] 16426 .IP \[bu] 2 16427 ISO8601 Date and time, local timezone - \f[C]2006-01-02T15:04:05\f[R] 16428 .IP \[bu] 2 16429 ISO8601 Date and time, local timezone - \f[C]2006-01-02 15:04:05\f[R] 16430 .IP \[bu] 2 16431 ISO8601 Date - \f[C]2006-01-02\f[R] (YYYY-MM-DD) 16432 .SS Size options 16433 .PP 16434 Options which use SIZE use KiB (multiples of 1024 bytes) by default. 16435 However, a suffix of \f[C]B\f[R] for Byte, \f[C]K\f[R] for KiB, 16436 \f[C]M\f[R] for MiB, \f[C]G\f[R] for GiB, \f[C]T\f[R] for TiB and 16437 \f[C]P\f[R] for PiB may be used. 16438 These are the binary units, e.g. 16439 1, 2**10, 2**20, 2**30 respectively. 16440 .SS --backup-dir=DIR 16441 .PP 16442 When using \f[C]sync\f[R], \f[C]copy\f[R] or \f[C]move\f[R] any files 16443 which would have been overwritten or deleted are moved in their original 16444 hierarchy into this directory. 16445 .PP 16446 If \f[C]--suffix\f[R] is set, then the moved files will have the suffix 16447 added to them. 16448 If there is a file with the same path (after the suffix has been added) 16449 in DIR, then it will be overwritten. 16450 .PP 16451 The remote in use must support server-side move or copy and you must use 16452 the same remote as the destination of the sync. 16453 The backup directory must not overlap the destination directory without 16454 it being excluded by a filter rule. 16455 .PP 16456 For example 16457 .IP 16458 .nf 16459 \f[C] 16460 rclone sync --interactive /path/to/local remote:current --backup-dir remote:old 16461 \f[R] 16462 .fi 16463 .PP 16464 will sync \f[C]/path/to/local\f[R] to \f[C]remote:current\f[R], but for 16465 any files which would have been updated or deleted will be stored in 16466 \f[C]remote:old\f[R]. 16467 .PP 16468 If running rclone from a script you might want to use today\[aq]s date 16469 as the directory name passed to \f[C]--backup-dir\f[R] to store the old 16470 files, or you might want to pass \f[C]--suffix\f[R] with today\[aq]s 16471 date. 16472 .PP 16473 See \f[C]--compare-dest\f[R] and \f[C]--copy-dest\f[R]. 16474 .SS --bind string 16475 .PP 16476 Local address to bind to for outgoing connections. 16477 This can be an IPv4 address (1.2.3.4), an IPv6 address (1234::789A) or 16478 host name. 16479 If the host name doesn\[aq]t resolve or resolves to more than one IP 16480 address it will give an error. 16481 .PP 16482 You can use \f[C]--bind 0.0.0.0\f[R] to force rclone to use IPv4 16483 addresses and \f[C]--bind ::0\f[R] to force rclone to use IPv6 16484 addresses. 16485 .SS --bwlimit=BANDWIDTH_SPEC 16486 .PP 16487 This option controls the bandwidth limit. 16488 For example 16489 .IP 16490 .nf 16491 \f[C] 16492 --bwlimit 10M 16493 \f[R] 16494 .fi 16495 .PP 16496 would mean limit the upload and download bandwidth to 10 MiB/s. 16497 \f[B]NB\f[R] this is \f[B]bytes\f[R] per second not \f[B]bits\f[R] per 16498 second. 16499 To use a single limit, specify the desired bandwidth in KiB/s, or use a 16500 suffix B|K|M|G|T|P. 16501 The default is \f[C]0\f[R] which means to not limit bandwidth. 16502 .PP 16503 The upload and download bandwidth can be specified separately, as 16504 \f[C]--bwlimit UP:DOWN\f[R], so 16505 .IP 16506 .nf 16507 \f[C] 16508 --bwlimit 10M:100k 16509 \f[R] 16510 .fi 16511 .PP 16512 would mean limit the upload bandwidth to 10 MiB/s and the download 16513 bandwidth to 100 KiB/s. 16514 Either limit can be \[dq]off\[dq] meaning no limit, so to just limit the 16515 upload bandwidth you would use 16516 .IP 16517 .nf 16518 \f[C] 16519 --bwlimit 10M:off 16520 \f[R] 16521 .fi 16522 .PP 16523 this would limit the upload bandwidth to 10 MiB/s but the download 16524 bandwidth would be unlimited. 16525 .PP 16526 When specified as above the bandwidth limits last for the duration of 16527 run of the rclone binary. 16528 .PP 16529 It is also possible to specify a \[dq]timetable\[dq] of limits, which 16530 will cause certain limits to be applied at certain times. 16531 To specify a timetable, format your entries as 16532 \f[C]WEEKDAY-HH:MM,BANDWIDTH WEEKDAY-HH:MM,BANDWIDTH...\f[R] where: 16533 \f[C]WEEKDAY\f[R] is optional element. 16534 .IP \[bu] 2 16535 \f[C]BANDWIDTH\f[R] can be a single number, e.g.\f[C]100k\f[R] or a pair 16536 of numbers for upload:download, e.g.\f[C]10M:1M\f[R]. 16537 .IP \[bu] 2 16538 \f[C]WEEKDAY\f[R] can be written as the whole word or only using the 16539 first 3 characters. 16540 It is optional. 16541 .IP \[bu] 2 16542 \f[C]HH:MM\f[R] is an hour from 00:00 to 23:59. 16543 .PP 16544 An example of a typical timetable to avoid link saturation during 16545 daytime working hours could be: 16546 .PP 16547 \f[C]--bwlimit \[dq]08:00,512k 12:00,10M 13:00,512k 18:00,30M 23:00,off\[dq]\f[R] 16548 .PP 16549 In this example, the transfer bandwidth will be set to 512 KiB/s at 8am 16550 every day. 16551 At noon, it will rise to 10 MiB/s, and drop back to 512 KiB/sec at 1pm. 16552 At 6pm, the bandwidth limit will be set to 30 MiB/s, and at 11pm it will 16553 be completely disabled (full speed). 16554 Anything between 11pm and 8am will remain unlimited. 16555 .PP 16556 An example of timetable with \f[C]WEEKDAY\f[R] could be: 16557 .PP 16558 \f[C]--bwlimit \[dq]Mon-00:00,512 Fri-23:59,10M Sat-10:00,1M Sun-20:00,off\[dq]\f[R] 16559 .PP 16560 It means that, the transfer bandwidth will be set to 512 KiB/s on 16561 Monday. 16562 It will rise to 10 MiB/s before the end of Friday. 16563 At 10:00 on Saturday it will be set to 1 MiB/s. 16564 From 20:00 on Sunday it will be unlimited. 16565 .PP 16566 Timeslots without \f[C]WEEKDAY\f[R] are extended to the whole week. 16567 So this example: 16568 .PP 16569 \f[C]--bwlimit \[dq]Mon-00:00,512 12:00,1M Sun-20:00,off\[dq]\f[R] 16570 .PP 16571 Is equivalent to this: 16572 .PP 16573 \f[C]--bwlimit \[dq]Mon-00:00,512Mon-12:00,1M Tue-12:00,1M Wed-12:00,1M Thu-12:00,1M Fri-12:00,1M Sat-12:00,1M Sun-12:00,1M Sun-20:00,off\[dq]\f[R] 16574 .PP 16575 Bandwidth limit apply to the data transfer for all backends. 16576 For most backends the directory listing bandwidth is also included 16577 (exceptions being the non HTTP backends, \f[C]ftp\f[R], \f[C]sftp\f[R] 16578 and \f[C]storj\f[R]). 16579 .PP 16580 Note that the units are \f[B]Byte/s\f[R], not \f[B]bit/s\f[R]. 16581 Typically connections are measured in bit/s - to convert divide by 8. 16582 For example, let\[aq]s say you have a 10 Mbit/s connection and you wish 16583 rclone to use half of it - 5 Mbit/s. 16584 This is 5/8 = 0.625 MiB/s so you would use a \f[C]--bwlimit 0.625M\f[R] 16585 parameter for rclone. 16586 .PP 16587 On Unix systems (Linux, macOS, \&...) the bandwidth limiter can be 16588 toggled by sending a \f[C]SIGUSR2\f[R] signal to rclone. 16589 This allows to remove the limitations of a long running rclone transfer 16590 and to restore it back to the value specified with \f[C]--bwlimit\f[R] 16591 quickly when needed. 16592 Assuming there is only one rclone instance running, you can toggle the 16593 limiter like this: 16594 .IP 16595 .nf 16596 \f[C] 16597 kill -SIGUSR2 $(pidof rclone) 16598 \f[R] 16599 .fi 16600 .PP 16601 If you configure rclone with a remote control then you can use change 16602 the bwlimit dynamically: 16603 .IP 16604 .nf 16605 \f[C] 16606 rclone rc core/bwlimit rate=1M 16607 \f[R] 16608 .fi 16609 .SS --bwlimit-file=BANDWIDTH_SPEC 16610 .PP 16611 This option controls per file bandwidth limit. 16612 For the options see the \f[C]--bwlimit\f[R] flag. 16613 .PP 16614 For example use this to allow no transfers to be faster than 1 MiB/s 16615 .IP 16616 .nf 16617 \f[C] 16618 --bwlimit-file 1M 16619 \f[R] 16620 .fi 16621 .PP 16622 This can be used in conjunction with \f[C]--bwlimit\f[R]. 16623 .PP 16624 Note that if a schedule is provided the file will use the schedule in 16625 effect at the start of the transfer. 16626 .SS --buffer-size=SIZE 16627 .PP 16628 Use this sized buffer to speed up file transfers. 16629 Each \f[C]--transfer\f[R] will use this much memory for buffering. 16630 .PP 16631 When using \f[C]mount\f[R] or \f[C]cmount\f[R] each open file descriptor 16632 will use this much memory for buffering. 16633 See the mount (https://rclone.org/commands/rclone_mount/#file-buffering) 16634 documentation for more details. 16635 .PP 16636 Set to \f[C]0\f[R] to disable the buffering for the minimum memory 16637 usage. 16638 .PP 16639 Note that the memory allocation of the buffers is influenced by the 16640 --use-mmap flag. 16641 .SS --cache-dir=DIR 16642 .PP 16643 Specify the directory rclone will use for caching, to override the 16644 default. 16645 .PP 16646 Default value is depending on operating system: - Windows 16647 \f[C]%LocalAppData%\[rs]rclone\f[R], if \f[C]LocalAppData\f[R] is 16648 defined. 16649 - macOS \f[C]$HOME/Library/Caches/rclone\f[R] if \f[C]HOME\f[R] is 16650 defined. 16651 - Unix \f[C]$XDG_CACHE_HOME/rclone\f[R] if \f[C]XDG_CACHE_HOME\f[R] is 16652 defined, else \f[C]$HOME/.cache/rclone\f[R] if \f[C]HOME\f[R] is 16653 defined. 16654 - Fallback (on all OS) to \f[C]$TMPDIR/rclone\f[R], where 16655 \f[C]TMPDIR\f[R] is the value from --temp-dir. 16656 .PP 16657 You can use the config 16658 paths (https://rclone.org/commands/rclone_config_paths/) command to see 16659 the current value. 16660 .PP 16661 Cache directory is heavily used by the VFS File 16662 Caching (https://rclone.org/commands/rclone_mount/#vfs-file-caching) 16663 mount feature, but also by 16664 serve (https://rclone.org/commands/rclone_serve/), GUI and other parts 16665 of rclone. 16666 .SS --check-first 16667 .PP 16668 If this flag is set then in a \f[C]sync\f[R], \f[C]copy\f[R] or 16669 \f[C]move\f[R], rclone will do all the checks to see whether files need 16670 to be transferred before doing any of the transfers. 16671 Normally rclone would start running transfers as soon as possible. 16672 .PP 16673 This flag can be useful on IO limited systems where transfers interfere 16674 with checking. 16675 .PP 16676 It can also be useful to ensure perfect ordering when using 16677 \f[C]--order-by\f[R]. 16678 .PP 16679 If both \f[C]--check-first\f[R] and \f[C]--order-by\f[R] are set when 16680 doing \f[C]rclone move\f[R] then rclone will use the transfer thread to 16681 delete source files which don\[aq]t need transferring. 16682 This will enable perfect ordering of the transfers and deletes but will 16683 cause the transfer stats to have more items in than expected. 16684 .PP 16685 Using this flag can use more memory as it effectively sets 16686 \f[C]--max-backlog\f[R] to infinite. 16687 This means that all the info on the objects to transfer is held in 16688 memory before the transfers start. 16689 .SS --checkers=N 16690 .PP 16691 Originally controlling just the number of file checkers to run in 16692 parallel, e.g. 16693 by \f[C]rclone copy\f[R]. 16694 Now a fairly universal parallelism control used by \f[C]rclone\f[R] in 16695 several places. 16696 .PP 16697 Note: checkers do the equality checking of files during a sync. 16698 For some storage systems (e.g. 16699 S3, Swift, Dropbox) this can take a significant amount of time so they 16700 are run in parallel. 16701 .PP 16702 The default is to run 8 checkers in parallel. 16703 However, in case of slow-reacting backends you may need to lower (rather 16704 than increase) this default by setting \f[C]--checkers\f[R] to 4 or less 16705 threads. 16706 This is especially advised if you are experiencing backend server 16707 crashes during file checking phase (e.g. 16708 on subsequent or top-up backups where little or no file copying is done 16709 and checking takes up most of the time). 16710 Increase this setting only with utmost care, while monitoring your 16711 server health and file checking throughput. 16712 .SS -c, --checksum 16713 .PP 16714 Normally rclone will look at modification time and size of files to see 16715 if they are equal. 16716 If you set this flag then rclone will check the file hash and size to 16717 determine if files are equal. 16718 .PP 16719 This is useful when the remote doesn\[aq]t support setting modified time 16720 and a more accurate sync is desired than just checking the file size. 16721 .PP 16722 This is very useful when transferring between remotes which store the 16723 same hash type on the object, e.g. 16724 Drive and Swift. 16725 For details of which remotes support which hash type see the table in 16726 the overview section (https://rclone.org/overview/). 16727 .PP 16728 Eg \f[C]rclone --checksum sync s3:/bucket swift:/bucket\f[R] would run 16729 much quicker than without the \f[C]--checksum\f[R] flag. 16730 .PP 16731 When using this flag, rclone won\[aq]t update mtimes of remote files if 16732 they are incorrect as it would normally. 16733 .SS --color WHEN 16734 .PP 16735 Specify when colors (and other ANSI codes) should be added to the 16736 output. 16737 .PP 16738 \f[C]AUTO\f[R] (default) only allows ANSI codes when the output is a 16739 terminal 16740 .PP 16741 \f[C]NEVER\f[R] never allow ANSI codes 16742 .PP 16743 \f[C]ALWAYS\f[R] always add ANSI codes, regardless of the output format 16744 (terminal or file) 16745 .SS --compare-dest=DIR 16746 .PP 16747 When using \f[C]sync\f[R], \f[C]copy\f[R] or \f[C]move\f[R] DIR is 16748 checked in addition to the destination for files. 16749 If a file identical to the source is found that file is NOT copied from 16750 source. 16751 This is useful to copy just files that have changed since the last 16752 backup. 16753 .PP 16754 You must use the same remote as the destination of the sync. 16755 The compare directory must not overlap the destination directory. 16756 .PP 16757 See \f[C]--copy-dest\f[R] and \f[C]--backup-dir\f[R]. 16758 .SS --config=CONFIG_FILE 16759 .PP 16760 Specify the location of the rclone configuration file, to override the 16761 default. 16762 E.g. 16763 \f[C]rclone config --config=\[dq]rclone.conf\[dq]\f[R]. 16764 .PP 16765 The exact default is a bit complex to describe, due to changes 16766 introduced through different versions of rclone while preserving 16767 backwards compatibility, but in most cases it is as simple as: 16768 .IP \[bu] 2 16769 \f[C]%APPDATA%/artpar/artpar.conf\f[R] on Windows 16770 .IP \[bu] 2 16771 \f[C]\[ti]/.config/artpar/artpar.conf\f[R] on other 16772 .PP 16773 The complete logic is as follows: Rclone will look for an existing 16774 configuration file in any of the following locations, in priority order: 16775 .IP "1." 3 16776 \f[C]rclone.conf\f[R] (in program directory, where rclone executable is) 16777 .IP "2." 3 16778 \f[C]%APPDATA%/artpar/artpar.conf\f[R] (only on Windows) 16779 .IP "3." 3 16780 \f[C]$XDG_CONFIG_HOME/artpar/artpar.conf\f[R] (on all systems, including 16781 Windows) 16782 .IP "4." 3 16783 \f[C]\[ti]/.config/artpar/artpar.conf\f[R] (see below for explanation of 16784 \[ti] symbol) 16785 .IP "5." 3 16786 \f[C]\[ti]/.rclone.conf\f[R] 16787 .PP 16788 If no existing configuration file is found, then a new one will be 16789 created in the following location: 16790 .IP \[bu] 2 16791 On Windows: Location 2 listed above, except in the unlikely event that 16792 \f[C]APPDATA\f[R] is not defined, then location 4 is used instead. 16793 .IP \[bu] 2 16794 On Unix: Location 3 if \f[C]XDG_CONFIG_HOME\f[R] is defined, else 16795 location 4. 16796 .IP \[bu] 2 16797 Fallback to location 5 (on all OS), when the rclone directory cannot be 16798 created, but if also a home directory was not found then path 16799 \f[C].rclone.conf\f[R] relative to current working directory will be 16800 used as a final resort. 16801 .PP 16802 The \f[C]\[ti]\f[R] symbol in paths above represent the home directory 16803 of the current user on any OS, and the value is defined as following: 16804 .IP \[bu] 2 16805 On Windows: \f[C]%HOME%\f[R] if defined, else \f[C]%USERPROFILE%\f[R], 16806 or else \f[C]%HOMEDRIVE%\[rs]%HOMEPATH%\f[R]. 16807 .IP \[bu] 2 16808 On Unix: \f[C]$HOME\f[R] if defined, else by looking up current user in 16809 OS-specific user database (e.g. 16810 passwd file), or else use the result from shell command 16811 \f[C]cd && pwd\f[R]. 16812 .PP 16813 If you run \f[C]rclone config file\f[R] you will see where the default 16814 location is for you. 16815 .PP 16816 The fact that an existing file \f[C]rclone.conf\f[R] in the same 16817 directory as the rclone executable is always preferred, means that it is 16818 easy to run in \[dq]portable\[dq] mode by downloading rclone executable 16819 to a writable directory and then create an empty file 16820 \f[C]rclone.conf\f[R] in the same directory. 16821 .PP 16822 If the location is set to empty string \f[C]\[dq]\[dq]\f[R] or path to a 16823 file with name \f[C]notfound\f[R], or the os null device represented by 16824 value \f[C]NUL\f[R] on Windows and \f[C]/dev/null\f[R] on Unix systems, 16825 then rclone will keep the config file in memory only. 16826 .PP 16827 The file format is basic 16828 INI (https://en.wikipedia.org/wiki/INI_file#Format): Sections of text, 16829 led by a \f[C][section]\f[R] header and followed by \f[C]key=value\f[R] 16830 entries on separate lines. 16831 In rclone each remote is represented by its own section, where the 16832 section name defines the name of the remote. 16833 Options are specified as the \f[C]key=value\f[R] entries, where the key 16834 is the option name without the \f[C]--backend-\f[R] prefix, in lowercase 16835 and with \f[C]_\f[R] instead of \f[C]-\f[R]. 16836 E.g. 16837 option \f[C]--mega-hard-delete\f[R] corresponds to key 16838 \f[C]hard_delete\f[R]. 16839 Only backend options can be specified. 16840 A special, and required, key \f[C]type\f[R] identifies the storage 16841 system (https://rclone.org/overview/), where the value is the internal 16842 lowercase name as returned by command \f[C]rclone help backends\f[R]. 16843 Comments are indicated by \f[C];\f[R] or \f[C]#\f[R] at the beginning of 16844 a line. 16845 .PP 16846 Example: 16847 .IP 16848 .nf 16849 \f[C] 16850 [megaremote] 16851 type = mega 16852 user = you\[at]example.com 16853 pass = PDPcQVVjVtzFY-GTdDFozqBhTdsPg3qH 16854 \f[R] 16855 .fi 16856 .PP 16857 Note that passwords are in 16858 obscured (https://rclone.org/commands/rclone_obscure/) form. 16859 Also, many storage systems uses token-based authentication instead of 16860 passwords, and this requires additional steps. 16861 It is easier, and safer, to use the interactive command 16862 \f[C]rclone config\f[R] instead of manually editing the configuration 16863 file. 16864 .PP 16865 The configuration file will typically contain login information, and 16866 should therefore have restricted permissions so that only the current 16867 user can read it. 16868 Rclone tries to ensure this when it writes the file. 16869 You may also choose to encrypt the file. 16870 .PP 16871 When token-based authentication are used, the configuration file must be 16872 writable, because rclone needs to update the tokens inside it. 16873 .PP 16874 To reduce risk of corrupting an existing configuration file, rclone will 16875 not write directly to it when saving changes. 16876 Instead it will first write to a new, temporary, file. 16877 If a configuration file already existed, it will (on Unix systems) try 16878 to mirror its permissions to the new file. 16879 Then it will rename the existing file to a temporary name as backup. 16880 Next, rclone will rename the new file to the correct name, before 16881 finally cleaning up by deleting the backup file. 16882 .PP 16883 If the configuration file path used by rclone is a symbolic link, then 16884 this will be evaluated and rclone will write to the resolved path, 16885 instead of overwriting the symbolic link. 16886 Temporary files used in the process (described above) will be written to 16887 the same parent directory as that of the resolved configuration file, 16888 but if this directory is also a symbolic link it will not be resolved 16889 and the temporary files will be written to the location of the directory 16890 symbolic link. 16891 .SS --contimeout=TIME 16892 .PP 16893 Set the connection timeout. 16894 This should be in go time format which looks like \f[C]5s\f[R] for 5 16895 seconds, \f[C]10m\f[R] for 10 minutes, or \f[C]3h30m\f[R]. 16896 .PP 16897 The connection timeout is the amount of time rclone will wait for a 16898 connection to go through to a remote object storage system. 16899 It is \f[C]1m\f[R] by default. 16900 .SS --copy-dest=DIR 16901 .PP 16902 When using \f[C]sync\f[R], \f[C]copy\f[R] or \f[C]move\f[R] DIR is 16903 checked in addition to the destination for files. 16904 If a file identical to the source is found that file is server-side 16905 copied from DIR to the destination. 16906 This is useful for incremental backup. 16907 .PP 16908 The remote in use must support server-side copy and you must use the 16909 same remote as the destination of the sync. 16910 The compare directory must not overlap the destination directory. 16911 .PP 16912 See \f[C]--compare-dest\f[R] and \f[C]--backup-dir\f[R]. 16913 .SS --dedupe-mode MODE 16914 .PP 16915 Mode to run dedupe command in. 16916 One of \f[C]interactive\f[R], \f[C]skip\f[R], \f[C]first\f[R], 16917 \f[C]newest\f[R], \f[C]oldest\f[R], \f[C]rename\f[R]. 16918 The default is \f[C]interactive\f[R]. 16919 .PD 0 16920 .P 16921 .PD 16922 See the dedupe command for more information as to what these options 16923 mean. 16924 .SS --default-time TIME 16925 .PP 16926 If a file or directory does have a modification time rclone can read 16927 then rclone will display this fixed time instead. 16928 .PP 16929 The default is \f[C]2000-01-01 00:00:00 UTC\f[R]. 16930 This can be configured in any of the ways shown in the time or duration 16931 options. 16932 .PP 16933 For example \f[C]--default-time 2020-06-01\f[R] to set the default time 16934 to the 1st of June 2020 or \f[C]--default-time 0s\f[R] to set the 16935 default time to the time rclone started up. 16936 .SS --disable FEATURE,FEATURE,... 16937 .PP 16938 This disables a comma separated list of optional features. 16939 For example to disable server-side move and server-side copy use: 16940 .IP 16941 .nf 16942 \f[C] 16943 --disable move,copy 16944 \f[R] 16945 .fi 16946 .PP 16947 The features can be put in any case. 16948 .PP 16949 To see a list of which features can be disabled use: 16950 .IP 16951 .nf 16952 \f[C] 16953 --disable help 16954 \f[R] 16955 .fi 16956 .PP 16957 The features a remote has can be seen in JSON format with: 16958 .IP 16959 .nf 16960 \f[C] 16961 rclone backend features remote: 16962 \f[R] 16963 .fi 16964 .PP 16965 See the overview features (https://rclone.org/overview/#features) and 16966 optional features (https://rclone.org/overview/#optional-features) to 16967 get an idea of which feature does what. 16968 .PP 16969 Note that some features can be set to \f[C]true\f[R] if they are 16970 \f[C]true\f[R]/\f[C]false\f[R] feature flag features by prefixing them 16971 with \f[C]!\f[R]. 16972 For example the \f[C]CaseInsensitive\f[R] feature can be forced to 16973 \f[C]false\f[R] with \f[C]--disable CaseInsensitive\f[R] and forced to 16974 \f[C]true\f[R] with \f[C]--disable \[aq]!CaseInsensitive\[aq]\f[R]. 16975 In general it isn\[aq]t a good idea doing this but it may be useful in 16976 extremis. 16977 .PP 16978 (Note that \f[C]!\f[R] is a shell command which you will need to escape 16979 with single quotes or a backslash on unix like platforms.) 16980 .PP 16981 This flag can be useful for debugging and in exceptional circumstances 16982 (e.g. 16983 Google Drive limiting the total volume of Server Side Copies to 100 16984 GiB/day). 16985 .SS --disable-http2 16986 .PP 16987 This stops rclone from trying to use HTTP/2 if available. 16988 This can sometimes speed up transfers due to a problem in the Go 16989 standard library (https://github.com/golang/go/issues/37373). 16990 .SS --dscp VALUE 16991 .PP 16992 Specify a DSCP value or name to use in connections. 16993 This could help QoS system to identify traffic class. 16994 BE, EF, DF, LE, CSx and AFxx are allowed. 16995 .PP 16996 See the description of differentiated 16997 services (https://en.wikipedia.org/wiki/Differentiated_services) to get 16998 an idea of this field. 16999 Setting this to 1 (LE) to identify the flow to SCAVENGER class can avoid 17000 occupying too much bandwidth in a network with DiffServ support (RFC 17001 8622 (https://tools.ietf.org/html/rfc8622)). 17002 .PP 17003 For example, if you configured QoS on router to handle LE properly. 17004 Running: 17005 .IP 17006 .nf 17007 \f[C] 17008 rclone copy --dscp LE from:/from to:/to 17009 \f[R] 17010 .fi 17011 .PP 17012 would make the priority lower than usual internet flows. 17013 .PP 17014 This option has no effect on Windows (see 17015 golang/go#42728 (https://github.com/golang/go/issues/42728)). 17016 .SS -n, --dry-run 17017 .PP 17018 Do a trial run with no permanent changes. 17019 Use this to see what rclone would do without actually doing it. 17020 Useful when setting up the \f[C]sync\f[R] command which deletes files in 17021 the destination. 17022 .SS --expect-continue-timeout=TIME 17023 .PP 17024 This specifies the amount of time to wait for a server\[aq]s first 17025 response headers after fully writing the request headers if the request 17026 has an \[dq]Expect: 100-continue\[dq] header. 17027 Not all backends support using this. 17028 .PP 17029 Zero means no timeout and causes the body to be sent immediately, 17030 without waiting for the server to approve. 17031 This time does not include the time to send the request header. 17032 .PP 17033 The default is \f[C]1s\f[R]. 17034 Set to \f[C]0\f[R] to disable. 17035 .SS --error-on-no-transfer 17036 .PP 17037 By default, rclone will exit with return code 0 if there were no errors. 17038 .PP 17039 This option allows rclone to return exit code 9 if no files were 17040 transferred between the source and destination. 17041 This allows using rclone in scripts, and triggering follow-on actions if 17042 data was copied, or skipping if not. 17043 .PP 17044 NB: Enabling this option turns a usually non-fatal error into a 17045 potentially fatal one - please check and adjust your scripts 17046 accordingly! 17047 .SS --fix-case 17048 .PP 17049 Normally, a sync to a case insensitive dest (such as macOS / Windows) 17050 will not result in a matching filename if the source and dest filenames 17051 have casing differences but are otherwise identical. 17052 For example, syncing \f[C]hello.txt\f[R] to \f[C]HELLO.txt\f[R] will 17053 normally result in the dest filename remaining \f[C]HELLO.txt\f[R]. 17054 If \f[C]--fix-case\f[R] is set, then \f[C]HELLO.txt\f[R] will be renamed 17055 to \f[C]hello.txt\f[R] to match the source. 17056 .PP 17057 NB: - directory names with incorrect casing will also be fixed - 17058 \f[C]--fix-case\f[R] will be ignored if \f[C]--immutable\f[R] is set - 17059 using \f[C]--local-case-sensitive\f[R] instead is not advisable; it will 17060 cause \f[C]HELLO.txt\f[R] to get deleted! - the old dest filename must 17061 not be excluded by filters. 17062 Be especially careful with 17063 \f[C]--files-from\f[R] (https://rclone.org/filtering/#files-from-read-list-of-source-file-names), 17064 which does not respect 17065 \f[C]--ignore-case\f[R] (https://rclone.org/filtering/#ignore-case-make-searches-case-insensitive)! 17066 - on remotes that do not support server-side move, \f[C]--fix-case\f[R] 17067 will require downloading the file and re-uploading it. 17068 To avoid this, do not use \f[C]--fix-case\f[R]. 17069 .SS --fs-cache-expire-duration=TIME 17070 .PP 17071 When using rclone via the API rclone caches created remotes for 5 17072 minutes by default in the \[dq]fs cache\[dq]. 17073 This means that if you do repeated actions on the same remote then 17074 rclone won\[aq]t have to build it again from scratch, which makes it 17075 more efficient. 17076 .PP 17077 This flag sets the time that the remotes are cached for. 17078 If you set it to \f[C]0\f[R] (or negative) then rclone won\[aq]t cache 17079 the remotes at all. 17080 .PP 17081 Note that if you use some flags, eg \f[C]--backup-dir\f[R] and if this 17082 is set to \f[C]0\f[R] rclone may build two remotes (one for the source 17083 or destination and one for the \f[C]--backup-dir\f[R] where it may have 17084 only built one before. 17085 .SS --fs-cache-expire-interval=TIME 17086 .PP 17087 This controls how often rclone checks for cached remotes to expire. 17088 See the \f[C]--fs-cache-expire-duration\f[R] documentation above for 17089 more info. 17090 The default is 60s, set to 0 to disable expiry. 17091 .SS --header 17092 .PP 17093 Add an HTTP header for all transactions. 17094 The flag can be repeated to add multiple headers. 17095 .PP 17096 If you want to add headers only for uploads use 17097 \f[C]--header-upload\f[R] and if you want to add headers only for 17098 downloads use \f[C]--header-download\f[R]. 17099 .PP 17100 This flag is supported for all HTTP based backends even those not 17101 supported by \f[C]--header-upload\f[R] and \f[C]--header-download\f[R] 17102 so may be used as a workaround for those with care. 17103 .IP 17104 .nf 17105 \f[C] 17106 rclone ls remote:test --header \[dq]X-Rclone: Foo\[dq] --header \[dq]X-LetMeIn: Yes\[dq] 17107 \f[R] 17108 .fi 17109 .SS --header-download 17110 .PP 17111 Add an HTTP header for all download transactions. 17112 The flag can be repeated to add multiple headers. 17113 .IP 17114 .nf 17115 \f[C] 17116 rclone sync --interactive s3:test/src \[ti]/dst --header-download \[dq]X-Amz-Meta-Test: Foo\[dq] --header-download \[dq]X-Amz-Meta-Test2: Bar\[dq] 17117 \f[R] 17118 .fi 17119 .PP 17120 See the GitHub issue here (https://github.com/artpar/artpar/issues/59) 17121 for currently supported backends. 17122 .SS --header-upload 17123 .PP 17124 Add an HTTP header for all upload transactions. 17125 The flag can be repeated to add multiple headers. 17126 .IP 17127 .nf 17128 \f[C] 17129 rclone sync --interactive \[ti]/src s3:test/dst --header-upload \[dq]Content-Disposition: attachment; filename=\[aq]cool.html\[aq]\[dq] --header-upload \[dq]X-Amz-Meta-Test: FooBar\[dq] 17130 \f[R] 17131 .fi 17132 .PP 17133 See the GitHub issue here (https://github.com/artpar/artpar/issues/59) 17134 for currently supported backends. 17135 .SS --human-readable 17136 .PP 17137 Rclone commands output values for sizes (e.g. 17138 number of bytes) and counts (e.g. 17139 number of files) either as \f[I]raw\f[R] numbers, or in 17140 \f[I]human-readable\f[R] format. 17141 .PP 17142 In human-readable format the values are scaled to larger units, 17143 indicated with a suffix shown after the value, and rounded to three 17144 decimals. 17145 Rclone consistently uses binary units (powers of 2) for sizes and 17146 decimal units (powers of 10) for counts. 17147 The unit prefix for size is according to IEC standard notation, e.g. 17148 \f[C]Ki\f[R] for kibi. 17149 Used with byte unit, \f[C]1 KiB\f[R] means 1024 Byte. 17150 In list type of output, only the unit prefix appended to the value (e.g. 17151 \f[C]9.762Ki\f[R]), while in more textual output the full unit is shown 17152 (e.g. 17153 \f[C]9.762 KiB\f[R]). 17154 For counts the SI standard notation is used, e.g. 17155 prefix \f[C]k\f[R] for kilo. 17156 Used with file counts, \f[C]1k\f[R] means 1000 files. 17157 .PP 17158 The various list (https://rclone.org/commands/rclone_ls/) commands 17159 output raw numbers by default. 17160 Option \f[C]--human-readable\f[R] will make them output values in 17161 human-readable format instead (with the short unit prefix). 17162 .PP 17163 The about (https://rclone.org/commands/rclone_about/) command outputs 17164 human-readable by default, with a command-specific option 17165 \f[C]--full\f[R] to output the raw numbers instead. 17166 .PP 17167 Command size (https://rclone.org/commands/rclone_size/) outputs both 17168 human-readable and raw numbers in the same output. 17169 .PP 17170 The tree (https://rclone.org/commands/rclone_tree/) command also 17171 considers \f[C]--human-readable\f[R], but it will not use the exact same 17172 notation as the other commands: It rounds to one decimal, and uses 17173 single letter suffix, e.g. 17174 \f[C]K\f[R] instead of \f[C]Ki\f[R]. 17175 The reason for this is that it relies on an external library. 17176 .PP 17177 The interactive command ncdu (https://rclone.org/commands/rclone_ncdu/) 17178 shows human-readable by default, and responds to key \f[C]u\f[R] for 17179 toggling human-readable format. 17180 .SS --ignore-case-sync 17181 .PP 17182 Using this option will cause rclone to ignore the case of the files when 17183 synchronizing so files will not be copied/synced when the existing 17184 filenames are the same, even if the casing is different. 17185 .SS --ignore-checksum 17186 .PP 17187 Normally rclone will check that the checksums of transferred files 17188 match, and give an error \[dq]corrupted on transfer\[dq] if they 17189 don\[aq]t. 17190 .PP 17191 You can use this option to skip that check. 17192 You should only use it if you have had the \[dq]corrupted on 17193 transfer\[dq] error message and you are sure you might want to transfer 17194 potentially corrupted data. 17195 .SS --ignore-existing 17196 .PP 17197 Using this option will make rclone unconditionally skip all files that 17198 exist on the destination, no matter the content of these files. 17199 .PP 17200 While this isn\[aq]t a generally recommended option, it can be useful in 17201 cases where your files change due to encryption. 17202 However, it cannot correct partial transfers in case a transfer was 17203 interrupted. 17204 .PP 17205 When performing a \f[C]move\f[R]/\f[C]moveto\f[R] command, this flag 17206 will leave skipped files in the source location unchanged when a file 17207 with the same name exists on the destination. 17208 .SS --ignore-size 17209 .PP 17210 Normally rclone will look at modification time and size of files to see 17211 if they are equal. 17212 If you set this flag then rclone will check only the modification time. 17213 If \f[C]--checksum\f[R] is set then it only checks the checksum. 17214 .PP 17215 It will also cause rclone to skip verifying the sizes are the same after 17216 transfer. 17217 .PP 17218 This can be useful for transferring files to and from OneDrive which 17219 occasionally misreports the size of image files (see 17220 #399 (https://github.com/artpar/artpar/issues/399) for more info). 17221 .SS -I, --ignore-times 17222 .PP 17223 Using this option will cause rclone to unconditionally upload all files 17224 regardless of the state of files on the destination. 17225 .PP 17226 Normally rclone would skip any files that have the same modification 17227 time and are the same size (or have the same checksum if using 17228 \f[C]--checksum\f[R]). 17229 .SS --immutable 17230 .PP 17231 Treat source and destination files as immutable and disallow 17232 modification. 17233 .PP 17234 With this option set, files will be created and deleted as requested, 17235 but existing files will never be updated. 17236 If an existing file does not match between the source and destination, 17237 rclone will give the error 17238 \f[C]Source and destination exist but do not match: immutable file modified\f[R]. 17239 .PP 17240 Note that only commands which transfer files (e.g. 17241 \f[C]sync\f[R], \f[C]copy\f[R], \f[C]move\f[R]) are affected by this 17242 behavior, and only modification is disallowed. 17243 Files may still be deleted explicitly (e.g. 17244 \f[C]delete\f[R], \f[C]purge\f[R]) or implicitly (e.g. 17245 \f[C]sync\f[R], \f[C]move\f[R]). 17246 Use \f[C]copy --immutable\f[R] if it is desired to avoid deletion as 17247 well as modification. 17248 .PP 17249 This can be useful as an additional layer of protection for immutable or 17250 append-only data sets (notably backup archives), where modification 17251 implies corruption and should not be propagated. 17252 .SS --inplace 17253 .PP 17254 The \f[C]--inplace\f[R] flag changes the behaviour of rclone when 17255 uploading files to some backends (backends with the 17256 \f[C]PartialUploads\f[R] feature flag set) such as: 17257 .IP \[bu] 2 17258 local 17259 .IP \[bu] 2 17260 ftp 17261 .IP \[bu] 2 17262 sftp 17263 .PP 17264 Without \f[C]--inplace\f[R] (the default) rclone will first upload to a 17265 temporary file with an extension like this, where \f[C]XXXXXX\f[R] 17266 represents a random string and \f[C].partial\f[R] is --partial-suffix 17267 value (\f[C].partial\f[R] by default). 17268 .IP 17269 .nf 17270 \f[C] 17271 original-file-name.XXXXXX.partial 17272 \f[R] 17273 .fi 17274 .PP 17275 (rclone will make sure the final name is no longer than 100 characters 17276 by truncating the \f[C]original-file-name\f[R] part if necessary). 17277 .PP 17278 When the upload is complete, rclone will rename the \f[C].partial\f[R] 17279 file to the correct name, overwriting any existing file at that point. 17280 If the upload fails then the \f[C].partial\f[R] file will be deleted. 17281 .PP 17282 This prevents other users of the backend from seeing partially uploaded 17283 files in their new names and prevents overwriting the old file until the 17284 new one is completely uploaded. 17285 .PP 17286 If the \f[C]--inplace\f[R] flag is supplied, rclone will upload directly 17287 to the final name without creating a \f[C].partial\f[R] file. 17288 .PP 17289 This means that an incomplete file will be visible in the directory 17290 listings while the upload is in progress and any existing files will be 17291 overwritten as soon as the upload starts. 17292 If the transfer fails then the file will be deleted. 17293 This can cause data loss of the existing file if the transfer fails. 17294 .PP 17295 Note that on the local file system if you don\[aq]t use 17296 \f[C]--inplace\f[R] hard links (Unix only) will be broken. 17297 And if you do use \f[C]--inplace\f[R] you won\[aq]t be able to update in 17298 use executables. 17299 .PP 17300 Note also that versions of rclone prior to v1.63.0 behave as if the 17301 \f[C]--inplace\f[R] flag is always supplied. 17302 .SS -i, --interactive 17303 .PP 17304 This flag can be used to tell rclone that you wish a manual confirmation 17305 before destructive operations. 17306 .PP 17307 It is \f[B]recommended\f[R] that you use this flag while learning rclone 17308 especially with \f[C]rclone sync\f[R]. 17309 .PP 17310 For example 17311 .IP 17312 .nf 17313 \f[C] 17314 $ rclone delete --interactive /tmp/dir 17315 rclone: delete \[dq]important-file.txt\[dq]? 17316 y) Yes, this is OK (default) 17317 n) No, skip this 17318 s) Skip all delete operations with no more questions 17319 !) Do all delete operations with no more questions 17320 q) Exit rclone now. 17321 y/n/s/!/q> n 17322 \f[R] 17323 .fi 17324 .PP 17325 The options mean 17326 .IP \[bu] 2 17327 \f[C]y\f[R]: \f[B]Yes\f[R], this operation should go ahead. 17328 You can also press Return for this to happen. 17329 You\[aq]ll be asked every time unless you choose \f[C]s\f[R] or 17330 \f[C]!\f[R]. 17331 .IP \[bu] 2 17332 \f[C]n\f[R]: \f[B]No\f[R], do not do this operation. 17333 You\[aq]ll be asked every time unless you choose \f[C]s\f[R] or 17334 \f[C]!\f[R]. 17335 .IP \[bu] 2 17336 \f[C]s\f[R]: \f[B]Skip\f[R] all the following operations of this type 17337 with no more questions. 17338 This takes effect until rclone exits. 17339 If there are any different kind of operations you\[aq]ll be prompted for 17340 them. 17341 .IP \[bu] 2 17342 \f[C]!\f[R]: \f[B]Do all\f[R] the following operations with no more 17343 questions. 17344 Useful if you\[aq]ve decided that you don\[aq]t mind rclone doing that 17345 kind of operation. 17346 This takes effect until rclone exits . 17347 If there are any different kind of operations you\[aq]ll be prompted for 17348 them. 17349 .IP \[bu] 2 17350 \f[C]q\f[R]: \f[B]Quit\f[R] rclone now, just in case! 17351 .SS --leave-root 17352 .PP 17353 During rmdirs it will not remove root directory, even if it\[aq]s empty. 17354 .SS --log-file=FILE 17355 .PP 17356 Log all of rclone\[aq]s output to FILE. 17357 This is not active by default. 17358 This can be useful for tracking down problems with syncs in combination 17359 with the \f[C]-v\f[R] flag. 17360 See the Logging section for more info. 17361 .PP 17362 If FILE exists then rclone will append to it. 17363 .PP 17364 Note that if you are using the \f[C]logrotate\f[R] program to manage 17365 rclone\[aq]s logs, then you should use the \f[C]copytruncate\f[R] option 17366 as rclone doesn\[aq]t have a signal to rotate logs. 17367 .SS --log-format LIST 17368 .PP 17369 Comma separated list of log format options. 17370 Accepted options are \f[C]date\f[R], \f[C]time\f[R], 17371 \f[C]microseconds\f[R], \f[C]pid\f[R], \f[C]longfile\f[R], 17372 \f[C]shortfile\f[R], \f[C]UTC\f[R]. 17373 Any other keywords will be silently ignored. 17374 \f[C]pid\f[R] will tag log messages with process identifier which useful 17375 with \f[C]rclone mount --daemon\f[R]. 17376 Other accepted options are explained in the go 17377 documentation (https://pkg.go.dev/log#pkg-constants). 17378 The default log format is \[dq]\f[C]date\f[R],\f[C]time\f[R]\[dq]. 17379 .SS --log-level LEVEL 17380 .PP 17381 This sets the log level for rclone. 17382 The default log level is \f[C]NOTICE\f[R]. 17383 .PP 17384 \f[C]DEBUG\f[R] is equivalent to \f[C]-vv\f[R]. 17385 It outputs lots of debug info - useful for bug reports and really 17386 finding out what rclone is doing. 17387 .PP 17388 \f[C]INFO\f[R] is equivalent to \f[C]-v\f[R]. 17389 It outputs information about each transfer and prints stats once a 17390 minute by default. 17391 .PP 17392 \f[C]NOTICE\f[R] is the default log level if no logging flags are 17393 supplied. 17394 It outputs very little when things are working normally. 17395 It outputs warnings and significant events. 17396 .PP 17397 \f[C]ERROR\f[R] is equivalent to \f[C]-q\f[R]. 17398 It only outputs error messages. 17399 .SS --use-json-log 17400 .PP 17401 This switches the log format to JSON for rclone. 17402 The fields of json log are level, msg, source, time. 17403 .SS --low-level-retries NUMBER 17404 .PP 17405 This controls the number of low level retries rclone does. 17406 .PP 17407 A low level retry is used to retry a failing operation - typically one 17408 HTTP request. 17409 This might be uploading a chunk of a big file for example. 17410 You will see low level retries in the log with the \f[C]-v\f[R] flag. 17411 .PP 17412 This shouldn\[aq]t need to be changed from the default in normal 17413 operations. 17414 However, if you get a lot of low level retries you may wish to reduce 17415 the value so rclone moves on to a high level retry (see the 17416 \f[C]--retries\f[R] flag) quicker. 17417 .PP 17418 Disable low level retries with \f[C]--low-level-retries 1\f[R]. 17419 .SS --max-backlog=N 17420 .PP 17421 This is the maximum allowable backlog of files in a sync/copy/move 17422 queued for being checked or transferred. 17423 .PP 17424 This can be set arbitrarily large. 17425 It will only use memory when the queue is in use. 17426 Note that it will use in the order of N KiB of memory when the backlog 17427 is in use. 17428 .PP 17429 Setting this large allows rclone to calculate how many files are pending 17430 more accurately, give a more accurate estimated finish time and make 17431 \f[C]--order-by\f[R] work more accurately. 17432 .PP 17433 Setting this small will make rclone more synchronous to the listings of 17434 the remote which may be desirable. 17435 .PP 17436 Setting this to a negative number will make the backlog as large as 17437 possible. 17438 .SS --max-delete=N 17439 .PP 17440 This tells rclone not to delete more than N files. 17441 If that limit is exceeded then a fatal error will be generated and 17442 rclone will stop the operation in progress. 17443 .SS --max-delete-size=SIZE 17444 .PP 17445 Rclone will stop deleting files when the total size of deletions has 17446 reached the size specified. 17447 It defaults to off. 17448 .PP 17449 If that limit is exceeded then a fatal error will be generated and 17450 rclone will stop the operation in progress. 17451 .SS --max-depth=N 17452 .PP 17453 This modifies the recursion depth for all the commands except purge. 17454 .PP 17455 So if you do \f[C]rclone --max-depth 1 ls remote:path\f[R] you will see 17456 only the files in the top level directory. 17457 Using \f[C]--max-depth 2\f[R] means you will see all the files in first 17458 two directory levels and so on. 17459 .PP 17460 For historical reasons the \f[C]lsd\f[R] command defaults to using a 17461 \f[C]--max-depth\f[R] of 1 - you can override this with the command line 17462 flag. 17463 .PP 17464 You can use this command to disable recursion (with 17465 \f[C]--max-depth 1\f[R]). 17466 .PP 17467 Note that if you use this with \f[C]sync\f[R] and 17468 \f[C]--delete-excluded\f[R] the files not recursed through are 17469 considered excluded and will be deleted on the destination. 17470 Test first with \f[C]--dry-run\f[R] if you are not sure what will 17471 happen. 17472 .SS --max-duration=TIME 17473 .PP 17474 Rclone will stop transferring when it has run for the duration 17475 specified. 17476 Defaults to off. 17477 .PP 17478 When the limit is reached all transfers will stop immediately. 17479 Use \f[C]--cutoff-mode\f[R] to modify this behaviour. 17480 .PP 17481 Rclone will exit with exit code 10 if the duration limit is reached. 17482 .SS --max-transfer=SIZE 17483 .PP 17484 Rclone will stop transferring when it has reached the size specified. 17485 Defaults to off. 17486 .PP 17487 When the limit is reached all transfers will stop immediately. 17488 Use \f[C]--cutoff-mode\f[R] to modify this behaviour. 17489 .PP 17490 Rclone will exit with exit code 8 if the transfer limit is reached. 17491 .SS --cutoff-mode=hard|soft|cautious 17492 .PP 17493 This modifies the behavior of \f[C]--max-transfer\f[R] and 17494 \f[C]--max-duration\f[R] Defaults to \f[C]--cutoff-mode=hard\f[R]. 17495 .PP 17496 Specifying \f[C]--cutoff-mode=hard\f[R] will stop transferring 17497 immediately when Rclone reaches the limit. 17498 .PP 17499 Specifying \f[C]--cutoff-mode=soft\f[R] will stop starting new transfers 17500 when Rclone reaches the limit. 17501 .PP 17502 Specifying \f[C]--cutoff-mode=cautious\f[R] will try to prevent Rclone 17503 from reaching the limit. 17504 Only applicable for \f[C]--max-transfer\f[R] 17505 .SS -M, --metadata 17506 .PP 17507 Setting this flag enables rclone to copy the metadata from the source to 17508 the destination. 17509 For local backends this is ownership, permissions, xattr etc. 17510 See the metadata section for more info. 17511 .SS --metadata-mapper SpaceSepList 17512 .PP 17513 If you supply the parameter \f[C]--metadata-mapper /path/to/program\f[R] 17514 then rclone will use that program to map metadata from source object to 17515 destination object. 17516 .PP 17517 The argument to this flag should be a command with an optional space 17518 separated list of arguments. 17519 If one of the arguments has a space in then enclose it in 17520 \f[C]\[dq]\f[R], if you want a literal \f[C]\[dq]\f[R] in an argument 17521 then enclose the argument in \f[C]\[dq]\f[R] and double the 17522 \f[C]\[dq]\f[R]. 17523 See CSV encoding (https://godoc.org/encoding/csv) for more info. 17524 .IP 17525 .nf 17526 \f[C] 17527 --metadata-mapper \[dq]python bin/test_metadata_mapper.py\[dq] 17528 --metadata-mapper \[aq]python bin/test_metadata_mapper.py \[dq]argument with a space\[dq]\[aq] 17529 --metadata-mapper \[aq]python bin/test_metadata_mapper.py \[dq]argument with \[dq]\[dq]two\[dq]\[dq] quotes\[dq]\[aq] 17530 \f[R] 17531 .fi 17532 .PP 17533 This uses a simple JSON based protocol with input on STDIN and output on 17534 STDOUT. 17535 This will be called for every file and directory copied and may be 17536 called concurrently. 17537 .PP 17538 The program\[aq]s job is to take a metadata blob on the input and turn 17539 it into a metadata blob on the output suitable for the destination 17540 backend. 17541 .PP 17542 Input to the program (via STDIN) might look like this. 17543 This provides some context for the \f[C]Metadata\f[R] which may be 17544 important. 17545 .IP \[bu] 2 17546 \f[C]SrcFs\f[R] is the config string for the remote that the object is 17547 currently on. 17548 .IP \[bu] 2 17549 \f[C]SrcFsType\f[R] is the name of the source backend. 17550 .IP \[bu] 2 17551 \f[C]DstFs\f[R] is the config string for the remote that the object is 17552 being copied to 17553 .IP \[bu] 2 17554 \f[C]DstFsType\f[R] is the name of the destination backend. 17555 .IP \[bu] 2 17556 \f[C]Remote\f[R] is the path of the object relative to the root. 17557 .IP \[bu] 2 17558 \f[C]Size\f[R], \f[C]MimeType\f[R], \f[C]ModTime\f[R] are attributes of 17559 the object. 17560 .IP \[bu] 2 17561 \f[C]IsDir\f[R] is \f[C]true\f[R] if this is a directory (not yet 17562 implemented). 17563 .IP \[bu] 2 17564 \f[C]ID\f[R] is the source \f[C]ID\f[R] of the object if known. 17565 .IP \[bu] 2 17566 \f[C]Metadata\f[R] is the backend specific metadata as described in the 17567 backend docs. 17568 .IP 17569 .nf 17570 \f[C] 17571 { 17572 \[dq]SrcFs\[dq]: \[dq]gdrive:\[dq], 17573 \[dq]SrcFsType\[dq]: \[dq]drive\[dq], 17574 \[dq]DstFs\[dq]: \[dq]newdrive:user\[dq], 17575 \[dq]DstFsType\[dq]: \[dq]onedrive\[dq], 17576 \[dq]Remote\[dq]: \[dq]test.txt\[dq], 17577 \[dq]Size\[dq]: 6, 17578 \[dq]MimeType\[dq]: \[dq]text/plain; charset=utf-8\[dq], 17579 \[dq]ModTime\[dq]: \[dq]2022-10-11T17:53:10.286745272+01:00\[dq], 17580 \[dq]IsDir\[dq]: false, 17581 \[dq]ID\[dq]: \[dq]xyz\[dq], 17582 \[dq]Metadata\[dq]: { 17583 \[dq]btime\[dq]: \[dq]2022-10-11T16:53:11Z\[dq], 17584 \[dq]content-type\[dq]: \[dq]text/plain; charset=utf-8\[dq], 17585 \[dq]mtime\[dq]: \[dq]2022-10-11T17:53:10.286745272+01:00\[dq], 17586 \[dq]owner\[dq]: \[dq]user1\[at]domain1.com\[dq], 17587 \[dq]permissions\[dq]: \[dq]...\[dq], 17588 \[dq]description\[dq]: \[dq]my nice file\[dq], 17589 \[dq]starred\[dq]: \[dq]false\[dq] 17590 } 17591 } 17592 \f[R] 17593 .fi 17594 .PP 17595 The program should then modify the input as desired and send it to 17596 STDOUT. 17597 The returned \f[C]Metadata\f[R] field will be used in its entirety for 17598 the destination object. 17599 Any other fields will be ignored. 17600 Note in this example we translate user names and permissions and add 17601 something to the description: 17602 .IP 17603 .nf 17604 \f[C] 17605 { 17606 \[dq]Metadata\[dq]: { 17607 \[dq]btime\[dq]: \[dq]2022-10-11T16:53:11Z\[dq], 17608 \[dq]content-type\[dq]: \[dq]text/plain; charset=utf-8\[dq], 17609 \[dq]mtime\[dq]: \[dq]2022-10-11T17:53:10.286745272+01:00\[dq], 17610 \[dq]owner\[dq]: \[dq]user1\[at]domain2.com\[dq], 17611 \[dq]permissions\[dq]: \[dq]...\[dq], 17612 \[dq]description\[dq]: \[dq]my nice file [migrated from domain1]\[dq], 17613 \[dq]starred\[dq]: \[dq]false\[dq] 17614 } 17615 } 17616 \f[R] 17617 .fi 17618 .PP 17619 Metadata can be removed here too. 17620 .PP 17621 An example python program might look something like this to implement 17622 the above transformations. 17623 .IP 17624 .nf 17625 \f[C] 17626 import sys, json 17627 17628 i = json.load(sys.stdin) 17629 metadata = i[\[dq]Metadata\[dq]] 17630 # Add tag to description 17631 if \[dq]description\[dq] in metadata: 17632 metadata[\[dq]description\[dq]] += \[dq] [migrated from domain1]\[dq] 17633 else: 17634 metadata[\[dq]description\[dq]] = \[dq][migrated from domain1]\[dq] 17635 # Modify owner 17636 if \[dq]owner\[dq] in metadata: 17637 metadata[\[dq]owner\[dq]] = metadata[\[dq]owner\[dq]].replace(\[dq]domain1.com\[dq], \[dq]domain2.com\[dq]) 17638 o = { \[dq]Metadata\[dq]: metadata } 17639 json.dump(o, sys.stdout, indent=\[dq]\[rs]t\[dq]) 17640 \f[R] 17641 .fi 17642 .PP 17643 You can find this example (slightly expanded) in the rclone source code 17644 at 17645 bin/test_metadata_mapper.py (https://github.com/artpar/artpar/blob/master/bin/test_metadata_mapper.py). 17646 .PP 17647 If you want to see the input to the metadata mapper and the output 17648 returned from it in the log you can use \f[C]-vv --dump mapper\f[R]. 17649 .PP 17650 See the metadata section for more info. 17651 .SS --metadata-set key=value 17652 .PP 17653 Add metadata \f[C]key\f[R] = \f[C]value\f[R] when uploading. 17654 This can be repeated as many times as required. 17655 See the metadata section for more info. 17656 .SS --modify-window=TIME 17657 .PP 17658 When checking whether a file has been modified, this is the maximum 17659 allowed time difference that a file can have and still be considered 17660 equivalent. 17661 .PP 17662 The default is \f[C]1ns\f[R] unless this is overridden by a remote. 17663 For example OS X only stores modification times to the nearest second so 17664 if you are reading and writing to an OS X filing system this will be 17665 \f[C]1s\f[R] by default. 17666 .PP 17667 This command line flag allows you to override that computed default. 17668 .SS --multi-thread-write-buffer-size=SIZE 17669 .PP 17670 When transferring with multiple threads, rclone will buffer SIZE bytes 17671 in memory before writing to disk for each thread. 17672 .PP 17673 This can improve performance if the underlying filesystem does not deal 17674 well with a lot of small writes in different positions of the file, so 17675 if you see transfers being limited by disk write speed, you might want 17676 to experiment with different values. 17677 Specially for magnetic drives and remote file systems a higher value can 17678 be useful. 17679 .PP 17680 Nevertheless, the default of \f[C]128k\f[R] should be fine for almost 17681 all use cases, so before changing it ensure that network is not really 17682 your bottleneck. 17683 .PP 17684 As a final hint, size is not the only factor: block size (or similar 17685 concept) can have an impact. 17686 In one case, we observed that exact multiples of 16k performed much 17687 better than other values. 17688 .SS --multi-thread-chunk-size=SizeSuffix 17689 .PP 17690 Normally the chunk size for multi thread transfers is set by the 17691 backend. 17692 However some backends such as \f[C]local\f[R] and \f[C]smb\f[R] (which 17693 implement \f[C]OpenWriterAt\f[R] but not \f[C]OpenChunkWriter\f[R]) 17694 don\[aq]t have a natural chunk size. 17695 .PP 17696 In this case the value of this option is used (default 64Mi). 17697 .SS --multi-thread-cutoff=SIZE 17698 .PP 17699 When transferring files above SIZE to capable backends, rclone will use 17700 multiple threads to transfer the file (default 256M). 17701 .PP 17702 Capable backends are marked in the 17703 overview (https://rclone.org/overview/#optional-features) as 17704 \f[C]MultithreadUpload\f[R]. 17705 (They need to implement either the \f[C]OpenWriterAt\f[R] or 17706 \f[C]OpenChunkWriter\f[R] internal interfaces). 17707 These include include, \f[C]local\f[R], \f[C]s3\f[R], 17708 \f[C]azureblob\f[R], \f[C]b2\f[R], \f[C]oracleobjectstorage\f[R] and 17709 \f[C]smb\f[R] at the time of writing. 17710 .PP 17711 On the local disk, rclone preallocates the file (using 17712 \f[C]fallocate(FALLOC_FL_KEEP_SIZE)\f[R] on unix or 17713 \f[C]NTSetInformationFile\f[R] on Windows both of which takes no time) 17714 then each thread writes directly into the file at the correct place. 17715 This means that rclone won\[aq]t create fragmented or sparse files and 17716 there won\[aq]t be any assembly time at the end of the transfer. 17717 .PP 17718 The number of threads used to transfer is controlled by 17719 \f[C]--multi-thread-streams\f[R]. 17720 .PP 17721 Use \f[C]-vv\f[R] if you wish to see info about the threads. 17722 .PP 17723 This will work with the \f[C]sync\f[R]/\f[C]copy\f[R]/\f[C]move\f[R] 17724 commands and friends \f[C]copyto\f[R]/\f[C]moveto\f[R]. 17725 Multi thread transfers will be used with \f[C]rclone mount\f[R] and 17726 \f[C]rclone serve\f[R] if \f[C]--vfs-cache-mode\f[R] is set to 17727 \f[C]writes\f[R] or above. 17728 .PP 17729 \f[B]NB\f[R] that this \f[B]only\f[R] works with supported backends as 17730 the destination but will work with any backend as the source. 17731 .PP 17732 \f[B]NB\f[R] that multi-thread copies are disabled for local to local 17733 copies as they are faster without unless 17734 \f[C]--multi-thread-streams\f[R] is set explicitly. 17735 .PP 17736 \f[B]NB\f[R] on Windows using multi-thread transfers to the local disk 17737 will cause the resulting files to be 17738 sparse (https://en.wikipedia.org/wiki/Sparse_file). 17739 Use \f[C]--local-no-sparse\f[R] to disable sparse files (which may cause 17740 long delays at the start of transfers) or disable multi-thread transfers 17741 with \f[C]--multi-thread-streams 0\f[R] 17742 .SS --multi-thread-streams=N 17743 .PP 17744 When using multi thread transfers (see above 17745 \f[C]--multi-thread-cutoff\f[R]) this sets the number of streams to use. 17746 Set to \f[C]0\f[R] to disable multi thread transfers (Default 4). 17747 .PP 17748 If the backend has a \f[C]--backend-upload-concurrency\f[R] setting (eg 17749 \f[C]--s3-upload-concurrency\f[R]) then this setting will be used as the 17750 number of transfers instead if it is larger than the value of 17751 \f[C]--multi-thread-streams\f[R] or \f[C]--multi-thread-streams\f[R] 17752 isn\[aq]t set. 17753 .SS --no-check-dest 17754 .PP 17755 The \f[C]--no-check-dest\f[R] can be used with \f[C]move\f[R] or 17756 \f[C]copy\f[R] and it causes rclone not to check the destination at all 17757 when copying files. 17758 .PP 17759 This means that: 17760 .IP \[bu] 2 17761 the destination is not listed minimising the API calls 17762 .IP \[bu] 2 17763 files are always transferred 17764 .IP \[bu] 2 17765 this can cause duplicates on remotes which allow it (e.g. 17766 Google Drive) 17767 .IP \[bu] 2 17768 \f[C]--retries 1\f[R] is recommended otherwise you\[aq]ll transfer 17769 everything again on a retry 17770 .PP 17771 This flag is useful to minimise the transactions if you know that none 17772 of the files are on the destination. 17773 .PP 17774 This is a specialized flag which should be ignored by most users! 17775 .SS --no-gzip-encoding 17776 .PP 17777 Don\[aq]t set \f[C]Accept-Encoding: gzip\f[R]. 17778 This means that rclone won\[aq]t ask the server for compressed files 17779 automatically. 17780 Useful if you\[aq]ve set the server to return files with 17781 \f[C]Content-Encoding: gzip\f[R] but you uploaded compressed files. 17782 .PP 17783 There is no need to set this in normal operation, and doing so will 17784 decrease the network transfer efficiency of rclone. 17785 .SS --no-traverse 17786 .PP 17787 The \f[C]--no-traverse\f[R] flag controls whether the destination file 17788 system is traversed when using the \f[C]copy\f[R] or \f[C]move\f[R] 17789 commands. 17790 \f[C]--no-traverse\f[R] is not compatible with \f[C]sync\f[R] and will 17791 be ignored if you supply it with \f[C]sync\f[R]. 17792 .PP 17793 If you are only copying a small number of files (or are filtering most 17794 of the files) and/or have a large number of files on the destination 17795 then \f[C]--no-traverse\f[R] will stop rclone listing the destination 17796 and save time. 17797 .PP 17798 However, if you are copying a large number of files, especially if you 17799 are doing a copy where lots of the files under consideration haven\[aq]t 17800 changed and won\[aq]t need copying then you shouldn\[aq]t use 17801 \f[C]--no-traverse\f[R]. 17802 .PP 17803 See rclone copy (https://rclone.org/commands/rclone_copy/) for an 17804 example of how to use it. 17805 .SS --no-unicode-normalization 17806 .PP 17807 Don\[aq]t normalize unicode characters in filenames during the sync 17808 routine. 17809 .PP 17810 Sometimes, an operating system will store filenames containing unicode 17811 parts in their decomposed form (particularly macOS). 17812 Some cloud storage systems will then recompose the unicode, resulting in 17813 duplicate files if the data is ever copied back to a local filesystem. 17814 .PP 17815 Using this flag will disable that functionality, treating each unicode 17816 character as unique. 17817 For example, by default e\[u0301] and \['e] will be normalized into the 17818 same character. 17819 With \f[C]--no-unicode-normalization\f[R] they will be treated as unique 17820 characters. 17821 .SS --no-update-modtime 17822 .PP 17823 When using this flag, rclone won\[aq]t update modification times of 17824 remote files if they are incorrect as it would normally. 17825 .PP 17826 This can be used if the remote is being synced with another tool also 17827 (e.g. 17828 the Google Drive client). 17829 .SS --no-update-dir-modtime 17830 .PP 17831 When using this flag, rclone won\[aq]t update modification times of 17832 remote directories if they are incorrect as it would normally. 17833 .SS --order-by string 17834 .PP 17835 The \f[C]--order-by\f[R] flag controls the order in which files in the 17836 backlog are processed in \f[C]rclone sync\f[R], \f[C]rclone copy\f[R] 17837 and \f[C]rclone move\f[R]. 17838 .PP 17839 The order by string is constructed like this. 17840 The first part describes what aspect is being measured: 17841 .IP \[bu] 2 17842 \f[C]size\f[R] - order by the size of the files 17843 .IP \[bu] 2 17844 \f[C]name\f[R] - order by the full path of the files 17845 .IP \[bu] 2 17846 \f[C]modtime\f[R] - order by the modification date of the files 17847 .PP 17848 This can have a modifier appended with a comma: 17849 .IP \[bu] 2 17850 \f[C]ascending\f[R] or \f[C]asc\f[R] - order so that the smallest (or 17851 oldest) is processed first 17852 .IP \[bu] 2 17853 \f[C]descending\f[R] or \f[C]desc\f[R] - order so that the largest (or 17854 newest) is processed first 17855 .IP \[bu] 2 17856 \f[C]mixed\f[R] - order so that the smallest is processed first for some 17857 threads and the largest for others 17858 .PP 17859 If the modifier is \f[C]mixed\f[R] then it can have an optional 17860 percentage (which defaults to \f[C]50\f[R]), e.g. 17861 \f[C]size,mixed,25\f[R] which means that 25% of the threads should be 17862 taking the smallest items and 75% the largest. 17863 The threads which take the smallest first will always take the smallest 17864 first and likewise the largest first threads. 17865 The \f[C]mixed\f[R] mode can be useful to minimise the transfer time 17866 when you are transferring a mixture of large and small files - the large 17867 files are guaranteed upload threads and bandwidth and the small files 17868 will be processed continuously. 17869 .PP 17870 If no modifier is supplied then the order is \f[C]ascending\f[R]. 17871 .PP 17872 For example 17873 .IP \[bu] 2 17874 \f[C]--order-by size,desc\f[R] - send the largest files first 17875 .IP \[bu] 2 17876 \f[C]--order-by modtime,ascending\f[R] - send the oldest files first 17877 .IP \[bu] 2 17878 \f[C]--order-by name\f[R] - send the files with alphabetically by path 17879 first 17880 .PP 17881 If the \f[C]--order-by\f[R] flag is not supplied or it is supplied with 17882 an empty string then the default ordering will be used which is as 17883 scanned. 17884 With \f[C]--checkers 1\f[R] this is mostly alphabetical, however with 17885 the default \f[C]--checkers 8\f[R] it is somewhat random. 17886 .SS Limitations 17887 .PP 17888 The \f[C]--order-by\f[R] flag does not do a separate pass over the data. 17889 This means that it may transfer some files out of the order specified if 17890 .IP \[bu] 2 17891 there are no files in the backlog or the source has not been fully 17892 scanned yet 17893 .IP \[bu] 2 17894 there are more than --max-backlog files in the backlog 17895 .PP 17896 Rclone will do its best to transfer the best file it has so in practice 17897 this should not cause a problem. 17898 Think of \f[C]--order-by\f[R] as being more of a best efforts flag 17899 rather than a perfect ordering. 17900 .PP 17901 If you want perfect ordering then you will need to specify --check-first 17902 which will find all the files which need transferring first before 17903 transferring any. 17904 .SS --partial-suffix 17905 .PP 17906 When --inplace is not used, it causes rclone to use the 17907 \f[C]--partial-suffix\f[R] as suffix for temporary files. 17908 .PP 17909 Suffix length limit is 16 characters. 17910 .PP 17911 The default is \f[C].partial\f[R]. 17912 .SS --password-command SpaceSepList 17913 .PP 17914 This flag supplies a program which should supply the config password 17915 when run. 17916 This is an alternative to rclone prompting for the password or setting 17917 the \f[C]RCLONE_CONFIG_PASS\f[R] variable. 17918 .PP 17919 The argument to this should be a command with a space separated list of 17920 arguments. 17921 If one of the arguments has a space in then enclose it in 17922 \f[C]\[dq]\f[R], if you want a literal \f[C]\[dq]\f[R] in an argument 17923 then enclose the argument in \f[C]\[dq]\f[R] and double the 17924 \f[C]\[dq]\f[R]. 17925 See CSV encoding (https://godoc.org/encoding/csv) for more info. 17926 .PP 17927 Eg 17928 .IP 17929 .nf 17930 \f[C] 17931 --password-command \[dq]echo hello\[dq] 17932 --password-command \[aq]echo \[dq]hello with space\[dq]\[aq] 17933 --password-command \[aq]echo \[dq]hello with \[dq]\[dq]quotes\[dq]\[dq] and space\[dq]\[aq] 17934 \f[R] 17935 .fi 17936 .PP 17937 See the Configuration Encryption for more info. 17938 .PP 17939 See a Windows PowerShell example on the 17940 Wiki (https://github.com/artpar/artpar/wiki/Windows-Powershell-use-rclone-password-command-for-Config-file-password). 17941 .SS -P, --progress 17942 .PP 17943 This flag makes rclone update the stats in a static block in the 17944 terminal providing a realtime overview of the transfer. 17945 .PP 17946 Any log messages will scroll above the static block. 17947 Log messages will push the static block down to the bottom of the 17948 terminal where it will stay. 17949 .PP 17950 Normally this is updated every 500mS but this period can be overridden 17951 with the \f[C]--stats\f[R] flag. 17952 .PP 17953 This can be used with the \f[C]--stats-one-line\f[R] flag for a simpler 17954 display. 17955 .PP 17956 Note: On Windows until this 17957 bug (https://github.com/Azure/go-ansiterm/issues/26) is fixed all 17958 non-ASCII characters will be replaced with \f[C].\f[R] when 17959 \f[C]--progress\f[R] is in use. 17960 .SS --progress-terminal-title 17961 .PP 17962 This flag, when used with \f[C]-P/--progress\f[R], will print the string 17963 \f[C]ETA: %s\f[R] to the terminal title. 17964 .SS -q, --quiet 17965 .PP 17966 This flag will limit rclone\[aq]s output to error messages only. 17967 .SS --refresh-times 17968 .PP 17969 The \f[C]--refresh-times\f[R] flag can be used to update modification 17970 times of existing files when they are out of sync on backends which 17971 don\[aq]t support hashes. 17972 .PP 17973 This is useful if you uploaded files with the incorrect timestamps and 17974 you now wish to correct them. 17975 .PP 17976 This flag is \f[B]only\f[R] useful for destinations which don\[aq]t 17977 support hashes (e.g. 17978 \f[C]crypt\f[R]). 17979 .PP 17980 This can be used any of the sync commands \f[C]sync\f[R], \f[C]copy\f[R] 17981 or \f[C]move\f[R]. 17982 .PP 17983 To use this flag you will need to be doing a modification time sync (so 17984 not using \f[C]--size-only\f[R] or \f[C]--checksum\f[R]). 17985 The flag will have no effect when using \f[C]--size-only\f[R] or 17986 \f[C]--checksum\f[R]. 17987 .PP 17988 If this flag is used when rclone comes to upload a file it will check to 17989 see if there is an existing file on the destination. 17990 If this file matches the source with size (and checksum if available) 17991 but has a differing timestamp then instead of re-uploading it, rclone 17992 will update the timestamp on the destination file. 17993 If the checksum does not match rclone will upload the new file. 17994 If the checksum is absent (e.g. 17995 on a \f[C]crypt\f[R] backend) then rclone will update the timestamp. 17996 .PP 17997 Note that some remotes can\[aq]t set the modification time without 17998 re-uploading the file so this flag is less useful on them. 17999 .PP 18000 Normally if you are doing a modification time sync rclone will update 18001 modification times without \f[C]--refresh-times\f[R] provided that the 18002 remote supports checksums \f[B]and\f[R] the checksums match on the file. 18003 However if the checksums are absent then rclone will upload the file 18004 rather than setting the timestamp as this is the safe behaviour. 18005 .SS --retries int 18006 .PP 18007 Retry the entire sync if it fails this many times it fails (default 3). 18008 .PP 18009 Some remotes can be unreliable and a few retries help pick up the files 18010 which didn\[aq]t get transferred because of errors. 18011 .PP 18012 Disable retries with \f[C]--retries 1\f[R]. 18013 .SS --retries-sleep=TIME 18014 .PP 18015 This sets the interval between each retry specified by 18016 \f[C]--retries\f[R] 18017 .PP 18018 The default is \f[C]0\f[R]. 18019 Use \f[C]0\f[R] to disable. 18020 .SS --server-side-across-configs 18021 .PP 18022 Allow server-side operations (e.g. 18023 copy or move) to work across different configurations. 18024 .PP 18025 This can be useful if you wish to do a server-side copy or move between 18026 two remotes which use the same backend but are configured differently. 18027 .PP 18028 Note that this isn\[aq]t enabled by default because it isn\[aq]t easy 18029 for rclone to tell if it will work between any two configurations. 18030 .SS --size-only 18031 .PP 18032 Normally rclone will look at modification time and size of files to see 18033 if they are equal. 18034 If you set this flag then rclone will check only the size. 18035 .PP 18036 This can be useful transferring files from Dropbox which have been 18037 modified by the desktop sync client which doesn\[aq]t set checksums of 18038 modification times in the same way as rclone. 18039 .SS --stats=TIME 18040 .PP 18041 Commands which transfer data (\f[C]sync\f[R], \f[C]copy\f[R], 18042 \f[C]copyto\f[R], \f[C]move\f[R], \f[C]moveto\f[R]) will print data 18043 transfer stats at regular intervals to show their progress. 18044 .PP 18045 This sets the interval. 18046 .PP 18047 The default is \f[C]1m\f[R]. 18048 Use \f[C]0\f[R] to disable. 18049 .PP 18050 If you set the stats interval then all commands can show stats. 18051 This can be useful when running other commands, \f[C]check\f[R] or 18052 \f[C]mount\f[R] for example. 18053 .PP 18054 Stats are logged at \f[C]INFO\f[R] level by default which means they 18055 won\[aq]t show at default log level \f[C]NOTICE\f[R]. 18056 Use \f[C]--stats-log-level NOTICE\f[R] or \f[C]-v\f[R] to make them 18057 show. 18058 See the Logging section for more info on log levels. 18059 .PP 18060 Note that on macOS you can send a SIGINFO (which is normally ctrl-T in 18061 the terminal) to make the stats print immediately. 18062 .SS --stats-file-name-length integer 18063 .PP 18064 By default, the \f[C]--stats\f[R] output will truncate file names and 18065 paths longer than 40 characters. 18066 This is equivalent to providing \f[C]--stats-file-name-length 40\f[R]. 18067 Use \f[C]--stats-file-name-length 0\f[R] to disable any truncation of 18068 file names printed by stats. 18069 .SS --stats-log-level string 18070 .PP 18071 Log level to show \f[C]--stats\f[R] output at. 18072 This can be \f[C]DEBUG\f[R], \f[C]INFO\f[R], \f[C]NOTICE\f[R], or 18073 \f[C]ERROR\f[R]. 18074 The default is \f[C]INFO\f[R]. 18075 This means at the default level of logging which is \f[C]NOTICE\f[R] the 18076 stats won\[aq]t show - if you want them to then use 18077 \f[C]--stats-log-level NOTICE\f[R]. 18078 See the Logging section for more info on log levels. 18079 .SS --stats-one-line 18080 .PP 18081 When this is specified, rclone condenses the stats into a single line 18082 showing the most important stats only. 18083 .SS --stats-one-line-date 18084 .PP 18085 When this is specified, rclone enables the single-line stats and 18086 prepends the display with a date string. 18087 The default is \f[C]2006/01/02 15:04:05 -\f[R] 18088 .SS --stats-one-line-date-format 18089 .PP 18090 When this is specified, rclone enables the single-line stats and 18091 prepends the display with a user-supplied date string. 18092 The date string MUST be enclosed in quotes. 18093 Follow golang specs (https://golang.org/pkg/time/#Time.Format) for date 18094 formatting syntax. 18095 .SS --stats-unit=bits|bytes 18096 .PP 18097 By default, data transfer rates will be printed in bytes per second. 18098 .PP 18099 This option allows the data rate to be printed in bits per second. 18100 .PP 18101 Data transfer volume will still be reported in bytes. 18102 .PP 18103 The rate is reported as a binary unit, not SI unit. 18104 So 1 Mbit/s equals 1,048,576 bit/s and not 1,000,000 bit/s. 18105 .PP 18106 The default is \f[C]bytes\f[R]. 18107 .SS --suffix=SUFFIX 18108 .PP 18109 When using \f[C]sync\f[R], \f[C]copy\f[R] or \f[C]move\f[R] any files 18110 which would have been overwritten or deleted will have the suffix added 18111 to them. 18112 If there is a file with the same path (after the suffix has been added), 18113 then it will be overwritten. 18114 .PP 18115 The remote in use must support server-side move or copy and you must use 18116 the same remote as the destination of the sync. 18117 .PP 18118 This is for use with files to add the suffix in the current directory or 18119 with \f[C]--backup-dir\f[R]. 18120 See \f[C]--backup-dir\f[R] for more info. 18121 .PP 18122 For example 18123 .IP 18124 .nf 18125 \f[C] 18126 rclone copy --interactive /path/to/local/file remote:current --suffix .bak 18127 \f[R] 18128 .fi 18129 .PP 18130 will copy \f[C]/path/to/local\f[R] to \f[C]remote:current\f[R], but for 18131 any files which would have been updated or deleted have .bak added. 18132 .PP 18133 If using \f[C]rclone sync\f[R] with \f[C]--suffix\f[R] and without 18134 \f[C]--backup-dir\f[R] then it is recommended to put a filter rule in 18135 excluding the suffix otherwise the \f[C]sync\f[R] will delete the backup 18136 files. 18137 .IP 18138 .nf 18139 \f[C] 18140 rclone sync --interactive /path/to/local/file remote:current --suffix .bak --exclude \[dq]*.bak\[dq] 18141 \f[R] 18142 .fi 18143 .SS --suffix-keep-extension 18144 .PP 18145 When using \f[C]--suffix\f[R], setting this causes rclone put the SUFFIX 18146 before the extension of the files that it backs up rather than after. 18147 .PP 18148 So let\[aq]s say we had \f[C]--suffix -2019-01-01\f[R], without the flag 18149 \f[C]file.txt\f[R] would be backed up to \f[C]file.txt-2019-01-01\f[R] 18150 and with the flag it would be backed up to 18151 \f[C]file-2019-01-01.txt\f[R]. 18152 This can be helpful to make sure the suffixed files can still be opened. 18153 .PP 18154 If a file has two (or more) extensions and the second (or subsequent) 18155 extension is recognised as a valid mime type, then the suffix will go 18156 before that extension. 18157 So \f[C]file.tar.gz\f[R] would be backed up to 18158 \f[C]file-2019-01-01.tar.gz\f[R] whereas \f[C]file.badextension.gz\f[R] 18159 would be backed up to \f[C]file.badextension-2019-01-01.gz\f[R]. 18160 .SS --syslog 18161 .PP 18162 On capable OSes (not Windows or Plan9) send all log output to syslog. 18163 .PP 18164 This can be useful for running rclone in a script or 18165 \f[C]rclone mount\f[R]. 18166 .SS --syslog-facility string 18167 .PP 18168 If using \f[C]--syslog\f[R] this sets the syslog facility (e.g. 18169 \f[C]KERN\f[R], \f[C]USER\f[R]). 18170 See \f[C]man syslog\f[R] for a list of possible facilities. 18171 The default facility is \f[C]DAEMON\f[R]. 18172 .SS --temp-dir=DIR 18173 .PP 18174 Specify the directory rclone will use for temporary files, to override 18175 the default. 18176 Make sure the directory exists and have accessible permissions. 18177 .PP 18178 By default the operating system\[aq]s temp directory will be used: - On 18179 Unix systems, \f[C]$TMPDIR\f[R] if non-empty, else \f[C]/tmp\f[R]. 18180 - On Windows, the first non-empty value from \f[C]%TMP%\f[R], 18181 \f[C]%TEMP%\f[R], \f[C]%USERPROFILE%\f[R], or the Windows directory. 18182 .PP 18183 When overriding the default with this option, the specified path will be 18184 set as value of environment variable \f[C]TMPDIR\f[R] on Unix systems 18185 and \f[C]TMP\f[R] and \f[C]TEMP\f[R] on Windows. 18186 .PP 18187 You can use the config 18188 paths (https://rclone.org/commands/rclone_config_paths/) command to see 18189 the current value. 18190 .SS --tpslimit float 18191 .PP 18192 Limit transactions per second to this number. 18193 Default is 0 which is used to mean unlimited transactions per second. 18194 .PP 18195 A transaction is roughly defined as an API call; its exact meaning will 18196 depend on the backend. 18197 For HTTP based backends it is an HTTP PUT/GET/POST/etc and its response. 18198 For FTP/SFTP it is a round trip transaction over TCP. 18199 .PP 18200 For example, to limit rclone to 10 transactions per second use 18201 \f[C]--tpslimit 10\f[R], or to 1 transaction every 2 seconds use 18202 \f[C]--tpslimit 0.5\f[R]. 18203 .PP 18204 Use this when the number of transactions per second from rclone is 18205 causing a problem with the cloud storage provider (e.g. 18206 getting you banned or rate limited). 18207 .PP 18208 This can be very useful for \f[C]rclone mount\f[R] to control the 18209 behaviour of applications using it. 18210 .PP 18211 This limit applies to all HTTP based backends and to the FTP and SFTP 18212 backends. 18213 It does not apply to the local backend or the Storj backend. 18214 .PP 18215 See also \f[C]--tpslimit-burst\f[R]. 18216 .SS --tpslimit-burst int 18217 .PP 18218 Max burst of transactions for \f[C]--tpslimit\f[R] (default 18219 \f[C]1\f[R]). 18220 .PP 18221 Normally \f[C]--tpslimit\f[R] will do exactly the number of transaction 18222 per second specified. 18223 However if you supply \f[C]--tps-burst\f[R] then rclone can save up some 18224 transactions from when it was idle giving a burst of up to the parameter 18225 supplied. 18226 .PP 18227 For example if you provide \f[C]--tpslimit-burst 10\f[R] then if rclone 18228 has been idle for more than 10*\f[C]--tpslimit\f[R] then it can do 10 18229 transactions very quickly before they are limited again. 18230 .PP 18231 This may be used to increase performance of \f[C]--tpslimit\f[R] without 18232 changing the long term average number of transactions per second. 18233 .SS --track-renames 18234 .PP 18235 By default, rclone doesn\[aq]t keep track of renamed files, so if you 18236 rename a file locally then sync it to a remote, rclone will delete the 18237 old file on the remote and upload a new copy. 18238 .PP 18239 An rclone sync with \f[C]--track-renames\f[R] runs like a normal sync, 18240 but keeps track of objects which exist in the destination but not in the 18241 source (which would normally be deleted), and which objects exist in the 18242 source but not the destination (which would normally be transferred). 18243 These objects are then candidates for renaming. 18244 .PP 18245 After the sync, rclone matches up the source only and destination only 18246 objects using the \f[C]--track-renames-strategy\f[R] specified and 18247 either renames the destination object or transfers the source and 18248 deletes the destination object. 18249 \f[C]--track-renames\f[R] is stateless like all of rclone\[aq]s syncs. 18250 .PP 18251 To use this flag the destination must support server-side copy or 18252 server-side move, and to use a hash based 18253 \f[C]--track-renames-strategy\f[R] (the default) the source and the 18254 destination must have a compatible hash. 18255 .PP 18256 If the destination does not support server-side copy or move, rclone 18257 will fall back to the default behaviour and log an error level message 18258 to the console. 18259 .PP 18260 Encrypted destinations are not currently supported by 18261 \f[C]--track-renames\f[R] if \f[C]--track-renames-strategy\f[R] includes 18262 \f[C]hash\f[R]. 18263 .PP 18264 Note that \f[C]--track-renames\f[R] is incompatible with 18265 \f[C]--no-traverse\f[R] and that it uses extra memory to keep track of 18266 all the rename candidates. 18267 .PP 18268 Note also that \f[C]--track-renames\f[R] is incompatible with 18269 \f[C]--delete-before\f[R] and will select \f[C]--delete-after\f[R] 18270 instead of \f[C]--delete-during\f[R]. 18271 .SS --track-renames-strategy (hash,modtime,leaf,size) 18272 .PP 18273 This option changes the file matching criteria for 18274 \f[C]--track-renames\f[R]. 18275 .PP 18276 The matching is controlled by a comma separated selection of these 18277 tokens: 18278 .IP \[bu] 2 18279 \f[C]modtime\f[R] - the modification time of the file - not supported on 18280 all backends 18281 .IP \[bu] 2 18282 \f[C]hash\f[R] - the hash of the file contents - not supported on all 18283 backends 18284 .IP \[bu] 2 18285 \f[C]leaf\f[R] - the name of the file not including its directory name 18286 .IP \[bu] 2 18287 \f[C]size\f[R] - the size of the file (this is always enabled) 18288 .PP 18289 The default option is \f[C]hash\f[R]. 18290 .PP 18291 Using \f[C]--track-renames-strategy modtime,leaf\f[R] would match files 18292 based on modification time, the leaf of the file name and the size only. 18293 .PP 18294 Using \f[C]--track-renames-strategy modtime\f[R] or \f[C]leaf\f[R] can 18295 enable \f[C]--track-renames\f[R] support for encrypted destinations. 18296 .PP 18297 Note that the \f[C]hash\f[R] strategy is not supported with encrypted 18298 destinations. 18299 .SS --delete-(before,during,after) 18300 .PP 18301 This option allows you to specify when files on your destination are 18302 deleted when you sync folders. 18303 .PP 18304 Specifying the value \f[C]--delete-before\f[R] will delete all files 18305 present on the destination, but not on the source \f[I]before\f[R] 18306 starting the transfer of any new or updated files. 18307 This uses two passes through the file systems, one for the deletions and 18308 one for the copies. 18309 .PP 18310 Specifying \f[C]--delete-during\f[R] will delete files while checking 18311 and uploading files. 18312 This is the fastest option and uses the least memory. 18313 .PP 18314 Specifying \f[C]--delete-after\f[R] (the default value) will delay 18315 deletion of files until all new/updated files have been successfully 18316 transferred. 18317 The files to be deleted are collected in the copy pass then deleted 18318 after the copy pass has completed successfully. 18319 The files to be deleted are held in memory so this mode may use more 18320 memory. 18321 This is the safest mode as it will only delete files if there have been 18322 no errors subsequent to that. 18323 If there have been errors before the deletions start then you will get 18324 the message \f[C]not deleting files as there were IO errors\f[R]. 18325 .SS --fast-list 18326 .PP 18327 When doing anything which involves a directory listing (e.g. 18328 \f[C]sync\f[R], \f[C]copy\f[R], \f[C]ls\f[R] - in fact nearly every 18329 command), rclone has different strategies to choose from. 18330 .PP 18331 The basic strategy is to list one directory and processes it before 18332 using more directory lists to process any subdirectories. 18333 This is a mandatory backend feature, called \f[C]List\f[R], which means 18334 it is supported by all backends. 18335 This strategy uses small amount of memory, and because it can be 18336 parallelised it is fast for operations involving processing of the list 18337 results. 18338 .PP 18339 Some backends provide the support for an alternative strategy, where all 18340 files beneath a directory can be listed in one (or a small number) of 18341 transactions. 18342 Rclone supports this alternative strategy through an optional backend 18343 feature called \f[C]ListR\f[R] (https://rclone.org/overview/#listr). 18344 You can see in the storage system overview documentation\[aq]s optional 18345 features (https://rclone.org/overview/#optional-features) section which 18346 backends it is enabled for (these tend to be the bucket-based ones, e.g. 18347 S3, B2, GCS, Swift). 18348 This strategy requires fewer transactions for highly recursive 18349 operations, which is important on backends where this is charged or 18350 heavily rate limited. 18351 It may be faster (due to fewer transactions) or slower (because it 18352 can\[aq]t be parallelized) depending on different parameters, and may 18353 require more memory if rclone has to keep the whole listing in memory. 18354 .PP 18355 Which listing strategy rclone picks for a given operation is 18356 complicated, but in general it tries to choose the best possible. 18357 It will prefer \f[C]ListR\f[R] in situations where it doesn\[aq]t need 18358 to store the listed files in memory, e.g. 18359 for unlimited recursive \f[C]ls\f[R] command variants. 18360 In other situations it will prefer \f[C]List\f[R], e.g. 18361 for \f[C]sync\f[R] and \f[C]copy\f[R], where it needs to keep the listed 18362 files in memory, and is performing operations on them where 18363 parallelization may be a huge advantage. 18364 .PP 18365 Rclone is not able to take all relevant parameters into account for 18366 deciding the best strategy, and therefore allows you to influence the 18367 choice in two ways: You can stop rclone from using \f[C]ListR\f[R] by 18368 disabling the feature, using the --disable option 18369 (\f[C]--disable ListR\f[R]), or you can allow rclone to use 18370 \f[C]ListR\f[R] where it would normally choose not to do so due to 18371 higher memory usage, using the \f[C]--fast-list\f[R] option. 18372 Rclone should always produce identical results either way. 18373 Using \f[C]--disable ListR\f[R] or \f[C]--fast-list\f[R] on a remote 18374 which doesn\[aq]t support \f[C]ListR\f[R] does nothing, rclone will just 18375 ignore it. 18376 .PP 18377 A rule of thumb is that if you pay for transactions and can fit your 18378 entire sync listing into memory, then \f[C]--fast-list\f[R] is 18379 recommended. 18380 If you have a very big sync to do, then don\[aq]t use 18381 \f[C]--fast-list\f[R], otherwise you will run out of memory. 18382 Run some tests and compare before you decide, and if in doubt then just 18383 leave the default, let rclone decide, i.e. 18384 not use \f[C]--fast-list\f[R]. 18385 .SS --timeout=TIME 18386 .PP 18387 This sets the IO idle timeout. 18388 If a transfer has started but then becomes idle for this long it is 18389 considered broken and disconnected. 18390 .PP 18391 The default is \f[C]5m\f[R]. 18392 Set to \f[C]0\f[R] to disable. 18393 .SS --transfers=N 18394 .PP 18395 The number of file transfers to run in parallel. 18396 It can sometimes be useful to set this to a smaller number if the remote 18397 is giving a lot of timeouts or bigger if you have lots of bandwidth and 18398 a fast remote. 18399 .PP 18400 The default is to run 4 file transfers in parallel. 18401 .PP 18402 Look at --multi-thread-streams if you would like to control single file 18403 transfers. 18404 .SS -u, --update 18405 .PP 18406 This forces rclone to skip any files which exist on the destination and 18407 have a modified time that is newer than the source file. 18408 .PP 18409 This can be useful in avoiding needless transfers when transferring to a 18410 remote which doesn\[aq]t support modification times directly (or when 18411 using \f[C]--use-server-modtime\f[R] to avoid extra API calls) as it is 18412 more accurate than a \f[C]--size-only\f[R] check and faster than using 18413 \f[C]--checksum\f[R]. 18414 On such remotes (or when using \f[C]--use-server-modtime\f[R]) the time 18415 checked will be the uploaded time. 18416 .PP 18417 If an existing destination file has a modification time older than the 18418 source file\[aq]s, it will be updated if the sizes are different. 18419 If the sizes are the same, it will be updated if the checksum is 18420 different or not available. 18421 .PP 18422 If an existing destination file has a modification time equal (within 18423 the computed modify window) to the source file\[aq]s, it will be updated 18424 if the sizes are different. 18425 The checksum will not be checked in this case unless the 18426 \f[C]--checksum\f[R] flag is provided. 18427 .PP 18428 In all other cases the file will not be updated. 18429 .PP 18430 Consider using the \f[C]--modify-window\f[R] flag to compensate for time 18431 skews between the source and the backend, for backends that do not 18432 support mod times, and instead use uploaded times. 18433 However, if the backend does not support checksums, note that syncing or 18434 copying within the time skew window may still result in additional 18435 transfers for safety. 18436 .SS --use-mmap 18437 .PP 18438 If this flag is set then rclone will use anonymous memory allocated by 18439 mmap on Unix based platforms and VirtualAlloc on Windows for its 18440 transfer buffers (size controlled by \f[C]--buffer-size\f[R]). 18441 Memory allocated like this does not go on the Go heap and can be 18442 returned to the OS immediately when it is finished with. 18443 .PP 18444 If this flag is not set then rclone will allocate and free the buffers 18445 using the Go memory allocator which may use more memory as memory pages 18446 are returned less aggressively to the OS. 18447 .PP 18448 It is possible this does not work well on all platforms so it is 18449 disabled by default; in the future it may be enabled by default. 18450 .SS --use-server-modtime 18451 .PP 18452 Some object-store backends (e.g, Swift, S3) do not preserve file 18453 modification times (modtime). 18454 On these backends, rclone stores the original modtime as additional 18455 metadata on the object. 18456 By default it will make an API call to retrieve the metadata when the 18457 modtime is needed by an operation. 18458 .PP 18459 Use this flag to disable the extra API call and rely instead on the 18460 server\[aq]s modified time. 18461 In cases such as a local to remote sync using \f[C]--update\f[R], 18462 knowing the local file is newer than the time it was last uploaded to 18463 the remote is sufficient. 18464 In those cases, this flag can speed up the process and reduce the number 18465 of API calls necessary. 18466 .PP 18467 Using this flag on a sync operation without also using 18468 \f[C]--update\f[R] would cause all files modified at any time other than 18469 the last upload time to be uploaded again, which is probably not what 18470 you want. 18471 .SS -v, -vv, --verbose 18472 .PP 18473 With \f[C]-v\f[R] rclone will tell you about each file that is 18474 transferred and a small number of significant events. 18475 .PP 18476 With \f[C]-vv\f[R] rclone will become very verbose telling you about 18477 every file it considers and transfers. 18478 Please send bug reports with a log with this setting. 18479 .PP 18480 When setting verbosity as an environment variable, use 18481 \f[C]RCLONE_VERBOSE=1\f[R] or \f[C]RCLONE_VERBOSE=2\f[R] for 18482 \f[C]-v\f[R] and \f[C]-vv\f[R] respectively. 18483 .SS -V, --version 18484 .PP 18485 Prints the version number 18486 .SS SSL/TLS options 18487 .PP 18488 The outgoing SSL/TLS connections rclone makes can be controlled with 18489 these options. 18490 For example this can be very useful with the HTTP or WebDAV backends. 18491 Rclone HTTP servers have their own set of configuration for SSL/TLS 18492 which you can find in their documentation. 18493 .SS --ca-cert stringArray 18494 .PP 18495 This loads the PEM encoded certificate authority certificates and uses 18496 it to verify the certificates of the servers rclone connects to. 18497 .PP 18498 If you have generated certificates signed with a local CA then you will 18499 need this flag to connect to servers using those certificates. 18500 .SS --client-cert string 18501 .PP 18502 This loads the PEM encoded client side certificate. 18503 .PP 18504 This is used for mutual TLS 18505 authentication (https://en.wikipedia.org/wiki/Mutual_authentication). 18506 .PP 18507 The \f[C]--client-key\f[R] flag is required too when using this. 18508 .SS --client-key string 18509 .PP 18510 This loads the PEM encoded client side private key used for mutual TLS 18511 authentication. 18512 Used in conjunction with \f[C]--client-cert\f[R]. 18513 .SS --no-check-certificate=true/false 18514 .PP 18515 \f[C]--no-check-certificate\f[R] controls whether a client verifies the 18516 server\[aq]s certificate chain and host name. 18517 If \f[C]--no-check-certificate\f[R] is true, TLS accepts any certificate 18518 presented by the server and any host name in that certificate. 18519 In this mode, TLS is susceptible to man-in-the-middle attacks. 18520 .PP 18521 This option defaults to \f[C]false\f[R]. 18522 .PP 18523 \f[B]This should be used only for testing.\f[R] 18524 .SS Configuration Encryption 18525 .PP 18526 Your configuration file contains information for logging in to your 18527 cloud services. 18528 This means that you should keep your \f[C]rclone.conf\f[R] file in a 18529 secure location. 18530 .PP 18531 If you are in an environment where that isn\[aq]t possible, you can add 18532 a password to your configuration. 18533 This means that you will have to supply the password every time you 18534 start rclone. 18535 .PP 18536 To add a password to your rclone configuration, execute 18537 \f[C]rclone config\f[R]. 18538 .IP 18539 .nf 18540 \f[C] 18541 >rclone config 18542 Current remotes: 18543 18544 e) Edit existing remote 18545 n) New remote 18546 d) Delete remote 18547 s) Set configuration password 18548 q) Quit config 18549 e/n/d/s/q> 18550 \f[R] 18551 .fi 18552 .PP 18553 Go into \f[C]s\f[R], Set configuration password: 18554 .IP 18555 .nf 18556 \f[C] 18557 e/n/d/s/q> s 18558 Your configuration is not encrypted. 18559 If you add a password, you will protect your login information to cloud services. 18560 a) Add Password 18561 q) Quit to main menu 18562 a/q> a 18563 Enter NEW configuration password: 18564 password: 18565 Confirm NEW password: 18566 password: 18567 Password set 18568 Your configuration is encrypted. 18569 c) Change Password 18570 u) Unencrypt configuration 18571 q) Quit to main menu 18572 c/u/q> 18573 \f[R] 18574 .fi 18575 .PP 18576 Your configuration is now encrypted, and every time you start rclone you 18577 will have to supply the password. 18578 See below for details. 18579 In the same menu, you can change the password or completely remove 18580 encryption from your configuration. 18581 .PP 18582 There is no way to recover the configuration if you lose your password. 18583 .PP 18584 rclone uses nacl 18585 secretbox (https://godoc.org/golang.org/x/crypto/nacl/secretbox) which 18586 in turn uses XSalsa20 and Poly1305 to encrypt and authenticate your 18587 configuration with secret-key cryptography. 18588 The password is SHA-256 hashed, which produces the key for secretbox. 18589 The hashed password is not stored. 18590 .PP 18591 While this provides very good security, we do not recommend storing your 18592 encrypted rclone configuration in public if it contains sensitive 18593 information, maybe except if you use a very strong password. 18594 .PP 18595 If it is safe in your environment, you can set the 18596 \f[C]RCLONE_CONFIG_PASS\f[R] environment variable to contain your 18597 password, in which case it will be used for decrypting the 18598 configuration. 18599 .PP 18600 You can set this for a session from a script. 18601 For unix like systems save this to a file called 18602 \f[C]set-rclone-password\f[R]: 18603 .IP 18604 .nf 18605 \f[C] 18606 #!/bin/echo Source this file don\[aq]t run it 18607 18608 read -s RCLONE_CONFIG_PASS 18609 export RCLONE_CONFIG_PASS 18610 \f[R] 18611 .fi 18612 .PP 18613 Then source the file when you want to use it. 18614 From the shell you would do \f[C]source set-rclone-password\f[R]. 18615 It will then ask you for the password and set it in the environment 18616 variable. 18617 .PP 18618 An alternate means of supplying the password is to provide a script 18619 which will retrieve the password and print on standard output. 18620 This script should have a fully specified path name and not rely on any 18621 environment variables. 18622 The script is supplied either via 18623 \f[C]--password-command=\[dq]...\[dq]\f[R] command line argument or via 18624 the \f[C]RCLONE_PASSWORD_COMMAND\f[R] environment variable. 18625 .PP 18626 One useful example of this is using the \f[C]passwordstore\f[R] 18627 application to retrieve the password: 18628 .IP 18629 .nf 18630 \f[C] 18631 export RCLONE_PASSWORD_COMMAND=\[dq]pass rclone/config\[dq] 18632 \f[R] 18633 .fi 18634 .PP 18635 If the \f[C]passwordstore\f[R] password manager holds the password for 18636 the rclone configuration, using the script method means the password is 18637 primarily protected by the \f[C]passwordstore\f[R] system, and is never 18638 embedded in the clear in scripts, nor available for examination using 18639 the standard commands available. 18640 It is quite possible with long running rclone sessions for copies of 18641 passwords to be innocently captured in log files or terminal scroll 18642 buffers, etc. 18643 Using the script method of supplying the password enhances the security 18644 of the config password considerably. 18645 .PP 18646 If you are running rclone inside a script, unless you are using the 18647 \f[C]--password-command\f[R] method, you might want to disable password 18648 prompts. 18649 To do that, pass the parameter \f[C]--ask-password=false\f[R] to rclone. 18650 This will make rclone fail instead of asking for a password if 18651 \f[C]RCLONE_CONFIG_PASS\f[R] doesn\[aq]t contain a valid password, and 18652 \f[C]--password-command\f[R] has not been supplied. 18653 .PP 18654 Whenever running commands that may be affected by options in a 18655 configuration file, rclone will look for an existing file according to 18656 the rules described above, and load any it finds. 18657 If an encrypted file is found, this includes decrypting it, with the 18658 possible consequence of a password prompt. 18659 When executing a command line that you know are not actually using 18660 anything from such a configuration file, you can avoid it being loaded 18661 by overriding the location, e.g. 18662 with one of the documented special values for memory-only configuration. 18663 Since only backend options can be stored in configuration files, this is 18664 normally unnecessary for commands that do not operate on backends, e.g. 18665 \f[C]genautocomplete\f[R]. 18666 However, it will be relevant for commands that do operate on backends in 18667 general, but are used without referencing a stored remote, e.g. 18668 listing local filesystem paths, or connection strings: 18669 \f[C]rclone --config=\[dq]\[dq] ls .\f[R] 18670 .SS Developer options 18671 .PP 18672 These options are useful when developing or debugging rclone. 18673 There are also some more remote specific options which aren\[aq]t 18674 documented here which are used for testing. 18675 These start with remote name e.g. 18676 \f[C]--drive-test-option\f[R] - see the docs for the remote in question. 18677 .SS --cpuprofile=FILE 18678 .PP 18679 Write CPU profile to file. 18680 This can be analysed with \f[C]go tool pprof\f[R]. 18681 .SS --dump flag,flag,flag 18682 .PP 18683 The \f[C]--dump\f[R] flag takes a comma separated list of flags to dump 18684 info about. 18685 .PP 18686 Note that some headers including \f[C]Accept-Encoding\f[R] as shown may 18687 not be correct in the request and the response may not show 18688 \f[C]Content-Encoding\f[R] if the go standard libraries auto gzip 18689 encoding was in effect. 18690 In this case the body of the request will be gunzipped before showing 18691 it. 18692 .PP 18693 The available flags are: 18694 .SS --dump headers 18695 .PP 18696 Dump HTTP headers with \f[C]Authorization:\f[R] lines removed. 18697 May still contain sensitive info. 18698 Can be very verbose. 18699 Useful for debugging only. 18700 .PP 18701 Use \f[C]--dump auth\f[R] if you do want the \f[C]Authorization:\f[R] 18702 headers. 18703 .SS --dump bodies 18704 .PP 18705 Dump HTTP headers and bodies - may contain sensitive info. 18706 Can be very verbose. 18707 Useful for debugging only. 18708 .PP 18709 Note that the bodies are buffered in memory so don\[aq]t use this for 18710 enormous files. 18711 .SS --dump requests 18712 .PP 18713 Like \f[C]--dump bodies\f[R] but dumps the request bodies and the 18714 response headers. 18715 Useful for debugging download problems. 18716 .SS --dump responses 18717 .PP 18718 Like \f[C]--dump bodies\f[R] but dumps the response bodies and the 18719 request headers. 18720 Useful for debugging upload problems. 18721 .SS --dump auth 18722 .PP 18723 Dump HTTP headers - will contain sensitive info such as 18724 \f[C]Authorization:\f[R] headers - use \f[C]--dump headers\f[R] to dump 18725 without \f[C]Authorization:\f[R] headers. 18726 Can be very verbose. 18727 Useful for debugging only. 18728 .SS --dump filters 18729 .PP 18730 Dump the filters to the output. 18731 Useful to see exactly what include and exclude options are filtering on. 18732 .SS --dump goroutines 18733 .PP 18734 This dumps a list of the running go-routines at the end of the command 18735 to standard output. 18736 .SS --dump openfiles 18737 .PP 18738 This dumps a list of the open files at the end of the command. 18739 It uses the \f[C]lsof\f[R] command to do that so you\[aq]ll need that 18740 installed to use it. 18741 .SS --dump mapper 18742 .PP 18743 This shows the JSON blobs being sent to the program supplied with 18744 \f[C]--metadata-mapper\f[R] and received from it. 18745 It can be useful for debugging the metadata mapper interface. 18746 .SS --memprofile=FILE 18747 .PP 18748 Write memory profile to file. 18749 This can be analysed with \f[C]go tool pprof\f[R]. 18750 .SS Filtering 18751 .PP 18752 For the filtering options 18753 .IP \[bu] 2 18754 \f[C]--delete-excluded\f[R] 18755 .IP \[bu] 2 18756 \f[C]--filter\f[R] 18757 .IP \[bu] 2 18758 \f[C]--filter-from\f[R] 18759 .IP \[bu] 2 18760 \f[C]--exclude\f[R] 18761 .IP \[bu] 2 18762 \f[C]--exclude-from\f[R] 18763 .IP \[bu] 2 18764 \f[C]--exclude-if-present\f[R] 18765 .IP \[bu] 2 18766 \f[C]--include\f[R] 18767 .IP \[bu] 2 18768 \f[C]--include-from\f[R] 18769 .IP \[bu] 2 18770 \f[C]--files-from\f[R] 18771 .IP \[bu] 2 18772 \f[C]--files-from-raw\f[R] 18773 .IP \[bu] 2 18774 \f[C]--min-size\f[R] 18775 .IP \[bu] 2 18776 \f[C]--max-size\f[R] 18777 .IP \[bu] 2 18778 \f[C]--min-age\f[R] 18779 .IP \[bu] 2 18780 \f[C]--max-age\f[R] 18781 .IP \[bu] 2 18782 \f[C]--dump filters\f[R] 18783 .IP \[bu] 2 18784 \f[C]--metadata-include\f[R] 18785 .IP \[bu] 2 18786 \f[C]--metadata-include-from\f[R] 18787 .IP \[bu] 2 18788 \f[C]--metadata-exclude\f[R] 18789 .IP \[bu] 2 18790 \f[C]--metadata-exclude-from\f[R] 18791 .IP \[bu] 2 18792 \f[C]--metadata-filter\f[R] 18793 .IP \[bu] 2 18794 \f[C]--metadata-filter-from\f[R] 18795 .PP 18796 See the filtering section (https://rclone.org/filtering/). 18797 .SS Remote control 18798 .PP 18799 For the remote control options and for instructions on how to remote 18800 control rclone 18801 .IP \[bu] 2 18802 \f[C]--rc\f[R] 18803 .IP \[bu] 2 18804 and anything starting with \f[C]--rc-\f[R] 18805 .PP 18806 See the remote control section (https://rclone.org/rc/). 18807 .SS Logging 18808 .PP 18809 rclone has 4 levels of logging, \f[C]ERROR\f[R], \f[C]NOTICE\f[R], 18810 \f[C]INFO\f[R] and \f[C]DEBUG\f[R]. 18811 .PP 18812 By default, rclone logs to standard error. 18813 This means you can redirect standard error and still see the normal 18814 output of rclone commands (e.g. 18815 \f[C]rclone ls\f[R]). 18816 .PP 18817 By default, rclone will produce \f[C]Error\f[R] and \f[C]Notice\f[R] 18818 level messages. 18819 .PP 18820 If you use the \f[C]-q\f[R] flag, rclone will only produce 18821 \f[C]Error\f[R] messages. 18822 .PP 18823 If you use the \f[C]-v\f[R] flag, rclone will produce \f[C]Error\f[R], 18824 \f[C]Notice\f[R] and \f[C]Info\f[R] messages. 18825 .PP 18826 If you use the \f[C]-vv\f[R] flag, rclone will produce \f[C]Error\f[R], 18827 \f[C]Notice\f[R], \f[C]Info\f[R] and \f[C]Debug\f[R] messages. 18828 .PP 18829 You can also control the log levels with the \f[C]--log-level\f[R] flag. 18830 .PP 18831 If you use the \f[C]--log-file=FILE\f[R] option, rclone will redirect 18832 \f[C]Error\f[R], \f[C]Info\f[R] and \f[C]Debug\f[R] messages along with 18833 standard error to FILE. 18834 .PP 18835 If you use the \f[C]--syslog\f[R] flag then rclone will log to syslog 18836 and the \f[C]--syslog-facility\f[R] control which facility it uses. 18837 .PP 18838 Rclone prefixes all log messages with their level in capitals, e.g. 18839 INFO which makes it easy to grep the log file for different kinds of 18840 information. 18841 .SS Exit Code 18842 .PP 18843 If any errors occur during the command execution, rclone will exit with 18844 a non-zero exit code. 18845 This allows scripts to detect when rclone operations have failed. 18846 .PP 18847 During the startup phase, rclone will exit immediately if an error is 18848 detected in the configuration. 18849 There will always be a log message immediately before exiting. 18850 .PP 18851 When rclone is running it will accumulate errors as it goes along, and 18852 only exit with a non-zero exit code if (after retries) there were still 18853 failed transfers. 18854 For every error counted there will be a high priority log message 18855 (visible with \f[C]-q\f[R]) showing the message and which file caused 18856 the problem. 18857 A high priority message is also shown when starting a retry so the user 18858 can see that any previous error messages may not be valid after the 18859 retry. 18860 If rclone has done a retry it will log a high priority message if the 18861 retry was successful. 18862 .SS List of exit codes 18863 .IP \[bu] 2 18864 \f[C]0\f[R] - success 18865 .IP \[bu] 2 18866 \f[C]1\f[R] - Syntax or usage error 18867 .IP \[bu] 2 18868 \f[C]2\f[R] - Error not otherwise categorised 18869 .IP \[bu] 2 18870 \f[C]3\f[R] - Directory not found 18871 .IP \[bu] 2 18872 \f[C]4\f[R] - File not found 18873 .IP \[bu] 2 18874 \f[C]5\f[R] - Temporary error (one that more retries might fix) (Retry 18875 errors) 18876 .IP \[bu] 2 18877 \f[C]6\f[R] - Less serious errors (like 461 errors from dropbox) 18878 (NoRetry errors) 18879 .IP \[bu] 2 18880 \f[C]7\f[R] - Fatal error (one that more retries won\[aq]t fix, like 18881 account suspended) (Fatal errors) 18882 .IP \[bu] 2 18883 \f[C]8\f[R] - Transfer exceeded - limit set by --max-transfer reached 18884 .IP \[bu] 2 18885 \f[C]9\f[R] - Operation successful, but no files transferred 18886 .IP \[bu] 2 18887 \f[C]10\f[R] - Duration exceeded - limit set by --max-duration reached 18888 .SS Environment Variables 18889 .PP 18890 Rclone can be configured entirely using environment variables. 18891 These can be used to set defaults for options or config file entries. 18892 .SS Options 18893 .PP 18894 Every option in rclone can have its default set by environment variable. 18895 .PP 18896 To find the name of the environment variable, first, take the long 18897 option name, strip the leading \f[C]--\f[R], change \f[C]-\f[R] to 18898 \f[C]_\f[R], make upper case and prepend \f[C]RCLONE_\f[R]. 18899 .PP 18900 For example, to always set \f[C]--stats 5s\f[R], set the environment 18901 variable \f[C]RCLONE_STATS=5s\f[R]. 18902 If you set stats on the command line this will override the environment 18903 variable setting. 18904 .PP 18905 Or to always use the trash in drive \f[C]--drive-use-trash\f[R], set 18906 \f[C]RCLONE_DRIVE_USE_TRASH=true\f[R]. 18907 .PP 18908 Verbosity is slightly different, the environment variable equivalent of 18909 \f[C]--verbose\f[R] or \f[C]-v\f[R] is \f[C]RCLONE_VERBOSE=1\f[R], or 18910 for \f[C]-vv\f[R], \f[C]RCLONE_VERBOSE=2\f[R]. 18911 .PP 18912 The same parser is used for the options and the environment variables so 18913 they take exactly the same form. 18914 .PP 18915 The options set by environment variables can be seen with the 18916 \f[C]-vv\f[R] flag, e.g. 18917 \f[C]rclone version -vv\f[R]. 18918 .SS Config file 18919 .PP 18920 You can set defaults for values in the config file on an individual 18921 remote basis. 18922 The names of the config items are documented in the page for each 18923 backend. 18924 .PP 18925 To find the name of the environment variable, you need to set, take 18926 \f[C]RCLONE_CONFIG_\f[R] + name of remote + \f[C]_\f[R] + name of config 18927 file option and make it all uppercase. 18928 Note one implication here is the remote\[aq]s name must be convertible 18929 into a valid environment variable name, so it can only contain letters, 18930 digits, or the \f[C]_\f[R] (underscore) character. 18931 .PP 18932 For example, to configure an S3 remote named \f[C]mys3:\f[R] without a 18933 config file (using unix ways of setting environment variables): 18934 .IP 18935 .nf 18936 \f[C] 18937 $ export RCLONE_CONFIG_MYS3_TYPE=s3 18938 $ export RCLONE_CONFIG_MYS3_ACCESS_KEY_ID=XXX 18939 $ export RCLONE_CONFIG_MYS3_SECRET_ACCESS_KEY=XXX 18940 $ rclone lsd mys3: 18941 -1 2016-09-21 12:54:21 -1 my-bucket 18942 $ rclone listremotes | grep mys3 18943 mys3: 18944 \f[R] 18945 .fi 18946 .PP 18947 Note that if you want to create a remote using environment variables you 18948 must create the \f[C]..._TYPE\f[R] variable as above. 18949 .PP 18950 Note that the name of a remote created using environment variable is 18951 case insensitive, in contrast to regular remotes stored in config file 18952 as documented above. 18953 You must write the name in uppercase in the environment variable, but as 18954 seen from example above it will be listed and can be accessed in 18955 lowercase, while you can also refer to the same remote in uppercase: 18956 .IP 18957 .nf 18958 \f[C] 18959 $ rclone lsd mys3: 18960 -1 2016-09-21 12:54:21 -1 my-bucket 18961 $ rclone lsd MYS3: 18962 -1 2016-09-21 12:54:21 -1 my-bucket 18963 \f[R] 18964 .fi 18965 .PP 18966 Note that you can only set the options of the immediate backend, so 18967 RCLONE_CONFIG_MYS3CRYPT_ACCESS_KEY_ID has no effect, if myS3Crypt is a 18968 crypt remote based on an S3 remote. 18969 However RCLONE_S3_ACCESS_KEY_ID will set the access key of all remotes 18970 using S3, including myS3Crypt. 18971 .PP 18972 Note also that now rclone has connection strings, it is probably easier 18973 to use those instead which makes the above example 18974 .IP 18975 .nf 18976 \f[C] 18977 rclone lsd :s3,access_key_id=XXX,secret_access_key=XXX: 18978 \f[R] 18979 .fi 18980 .SS Precedence 18981 .PP 18982 The various different methods of backend configuration are read in this 18983 order and the first one with a value is used. 18984 .IP \[bu] 2 18985 Parameters in connection strings, e.g. 18986 \f[C]myRemote,skip_links:\f[R] 18987 .IP \[bu] 2 18988 Flag values as supplied on the command line, e.g. 18989 \f[C]--skip-links\f[R] 18990 .IP \[bu] 2 18991 Remote specific environment vars, e.g. 18992 \f[C]RCLONE_CONFIG_MYREMOTE_SKIP_LINKS\f[R] (see above). 18993 .IP \[bu] 2 18994 Backend-specific environment vars, e.g. 18995 \f[C]RCLONE_LOCAL_SKIP_LINKS\f[R]. 18996 .IP \[bu] 2 18997 Backend generic environment vars, e.g. 18998 \f[C]RCLONE_SKIP_LINKS\f[R]. 18999 .IP \[bu] 2 19000 Config file, e.g. 19001 \f[C]skip_links = true\f[R]. 19002 .IP \[bu] 2 19003 Default values, e.g. 19004 \f[C]false\f[R] - these can\[aq]t be changed. 19005 .PP 19006 So if both \f[C]--skip-links\f[R] is supplied on the command line and an 19007 environment variable \f[C]RCLONE_LOCAL_SKIP_LINKS\f[R] is set, the 19008 command line flag will take preference. 19009 .PP 19010 The backend configurations set by environment variables can be seen with 19011 the \f[C]-vv\f[R] flag, e.g. 19012 \f[C]rclone about myRemote: -vv\f[R]. 19013 .PP 19014 For non backend configuration the order is as follows: 19015 .IP \[bu] 2 19016 Flag values as supplied on the command line, e.g. 19017 \f[C]--stats 5s\f[R]. 19018 .IP \[bu] 2 19019 Environment vars, e.g. 19020 \f[C]RCLONE_STATS=5s\f[R]. 19021 .IP \[bu] 2 19022 Default values, e.g. 19023 \f[C]1m\f[R] - these can\[aq]t be changed. 19024 .SS Other environment variables 19025 .IP \[bu] 2 19026 \f[C]RCLONE_CONFIG_PASS\f[R] set to contain your config file password 19027 (see Configuration Encryption section) 19028 .IP \[bu] 2 19029 \f[C]HTTP_PROXY\f[R], \f[C]HTTPS_PROXY\f[R] and \f[C]NO_PROXY\f[R] (or 19030 the lowercase versions thereof). 19031 .RS 2 19032 .IP \[bu] 2 19033 \f[C]HTTPS_PROXY\f[R] takes precedence over \f[C]HTTP_PROXY\f[R] for 19034 https requests. 19035 .IP \[bu] 2 19036 The environment values may be either a complete URL or a 19037 \[dq]host[:port]\[dq] for, in which case the \[dq]http\[dq] scheme is 19038 assumed. 19039 .RE 19040 .IP \[bu] 2 19041 \f[C]USER\f[R] and \f[C]LOGNAME\f[R] values are used as fallbacks for 19042 current username. 19043 The primary method for looking up username is OS-specific: Windows API 19044 on Windows, real user ID in /etc/passwd on Unix systems. 19045 In the documentation the current username is simply referred to as 19046 \f[C]$USER\f[R]. 19047 .IP \[bu] 2 19048 \f[C]RCLONE_CONFIG_DIR\f[R] - rclone \f[B]sets\f[R] this variable for 19049 use in config files and sub processes to point to the directory holding 19050 the config file. 19051 .PP 19052 The options set by environment variables can be seen with the 19053 \f[C]-vv\f[R] and \f[C]--log-level=DEBUG\f[R] flags, e.g. 19054 \f[C]rclone version -vv\f[R]. 19055 .SH Configuring rclone on a remote / headless machine 19056 .PP 19057 Some of the configurations (those involving oauth2) require an Internet 19058 connected web browser. 19059 .PP 19060 If you are trying to set rclone up on a remote or headless box with no 19061 browser available on it (e.g. 19062 a NAS or a server in a datacenter) then you will need to use an 19063 alternative means of configuration. 19064 There are two ways of doing it, described below. 19065 .SS Configuring using rclone authorize 19066 .PP 19067 On the headless box run \f[C]rclone\f[R] config but answer \f[C]N\f[R] 19068 to the \f[C]Use web browser to automatically authenticate?\f[R] 19069 question. 19070 .IP 19071 .nf 19072 \f[C] 19073 \&... 19074 Remote config 19075 Use web browser to automatically authenticate rclone with remote? 19076 * Say Y if the machine running rclone has a web browser you can use 19077 * Say N if running rclone on a (remote) machine without web browser access 19078 If not sure try Y. If Y failed, try N. 19079 y) Yes (default) 19080 n) No 19081 y/n> n 19082 For this to work, you will need rclone available on a machine that has 19083 a web browser available. 19084 19085 For more help and alternate methods see: https://rclone.org/remote_setup/ 19086 19087 Execute the following on the machine with the web browser (same rclone 19088 version recommended): 19089 19090 rclone authorize \[dq]dropbox\[dq] 19091 19092 Then paste the result below: 19093 result> 19094 \f[R] 19095 .fi 19096 .PP 19097 Then on your main desktop machine 19098 .IP 19099 .nf 19100 \f[C] 19101 rclone authorize \[dq]dropbox\[dq] 19102 If your browser doesn\[aq]t open automatically go to the following link: http://127.0.0.1:53682/auth 19103 Log in and authorize rclone for access 19104 Waiting for code... 19105 Got code 19106 Paste the following into your remote machine ---> 19107 SECRET_TOKEN 19108 <---End paste 19109 \f[R] 19110 .fi 19111 .PP 19112 Then back to the headless box, paste in the code 19113 .IP 19114 .nf 19115 \f[C] 19116 result> SECRET_TOKEN 19117 -------------------- 19118 [acd12] 19119 client_id = 19120 client_secret = 19121 token = SECRET_TOKEN 19122 -------------------- 19123 y) Yes this is OK 19124 e) Edit this remote 19125 d) Delete this remote 19126 y/e/d> 19127 \f[R] 19128 .fi 19129 .SS Configuring by copying the config file 19130 .PP 19131 Rclone stores all of its config in a single configuration file. 19132 This can easily be copied to configure a remote rclone. 19133 .PP 19134 So first configure rclone on your desktop machine with 19135 .IP 19136 .nf 19137 \f[C] 19138 rclone config 19139 \f[R] 19140 .fi 19141 .PP 19142 to set up the config file. 19143 .PP 19144 Find the config file by running \f[C]rclone config file\f[R], for 19145 example 19146 .IP 19147 .nf 19148 \f[C] 19149 $ rclone config file 19150 Configuration file is stored at: 19151 /home/user/.rclone.conf 19152 \f[R] 19153 .fi 19154 .PP 19155 Now transfer it to the remote box (scp, cut paste, ftp, sftp, etc.) and 19156 place it in the correct place (use \f[C]rclone config file\f[R] on the 19157 remote box to find out where). 19158 .SS Configuring using SSH Tunnel 19159 .PP 19160 Linux and MacOS users can utilize SSH Tunnel to redirect the headless 19161 box port 53682 to local machine by using the following command: 19162 .IP 19163 .nf 19164 \f[C] 19165 ssh -L localhost:53682:localhost:53682 username\[at]remote_server 19166 \f[R] 19167 .fi 19168 .PP 19169 Then on the headless box run \f[C]rclone\f[R] config and answer 19170 \f[C]Y\f[R] to the 19171 \f[C]Use web browser to automatically authenticate?\f[R] question. 19172 .IP 19173 .nf 19174 \f[C] 19175 \&... 19176 Remote config 19177 Use web browser to automatically authenticate rclone with remote? 19178 * Say Y if the machine running rclone has a web browser you can use 19179 * Say N if running rclone on a (remote) machine without web browser access 19180 If not sure try Y. If Y failed, try N. 19181 y) Yes (default) 19182 n) No 19183 y/n> y 19184 \f[R] 19185 .fi 19186 .PP 19187 Then copy and paste the auth url 19188 \f[C]http://127.0.0.1:53682/auth?state=xxxxxxxxxxxx\f[R] to the browser 19189 on your local machine, complete the auth and it is done. 19190 .SH Filtering, includes and excludes 19191 .PP 19192 Filter flags determine which files rclone \f[C]sync\f[R], 19193 \f[C]move\f[R], \f[C]ls\f[R], \f[C]lsl\f[R], \f[C]md5sum\f[R], 19194 \f[C]sha1sum\f[R], \f[C]size\f[R], \f[C]delete\f[R], \f[C]check\f[R] and 19195 similar commands apply to. 19196 .PP 19197 They are specified in terms of path/file name patterns; path/file lists; 19198 file age and size, or presence of a file in a directory. 19199 Bucket based remotes without the concept of directory apply filters to 19200 object key, age and size in an analogous way. 19201 .PP 19202 Rclone \f[C]purge\f[R] does not obey filters. 19203 .PP 19204 To test filters without risk of damage to data, apply them to 19205 \f[C]rclone ls\f[R], or with the \f[C]--dry-run\f[R] and \f[C]-vv\f[R] 19206 flags. 19207 .PP 19208 Rclone filter patterns can only be used in filter command line options, 19209 not in the specification of a remote. 19210 .PP 19211 E.g. 19212 \f[C]rclone copy \[dq]remote:dir*.jpg\[dq] /path/to/dir\f[R] does not 19213 have a filter effect. 19214 \f[C]rclone copy remote:dir /path/to/dir --include \[dq]*.jpg\[dq]\f[R] 19215 does. 19216 .PP 19217 \f[B]Important\f[R] Avoid mixing any two of \f[C]--include...\f[R], 19218 \f[C]--exclude...\f[R] or \f[C]--filter...\f[R] flags in an rclone 19219 command. 19220 The results might not be what you expect. 19221 Instead use a \f[C]--filter...\f[R] flag. 19222 .SS Patterns for matching path/file names 19223 .SS Pattern syntax 19224 .PP 19225 Here is a formal definition of the pattern syntax, examples are below. 19226 .PP 19227 Rclone matching rules follow a glob style: 19228 .IP 19229 .nf 19230 \f[C] 19231 * matches any sequence of non-separator (/) characters 19232 ** matches any sequence of characters including / separators 19233 ? matches any single non-separator (/) character 19234 [ [ ! ] { character-range } ] 19235 character class (must be non-empty) 19236 { pattern-list } 19237 pattern alternatives 19238 {{ regexp }} 19239 regular expression to match 19240 c matches character c (c != *, **, ?, \[rs], [, {, }) 19241 \[rs]c matches reserved character c (c = *, **, ?, \[rs], [, {, }) or character class 19242 \f[R] 19243 .fi 19244 .PP 19245 character-range: 19246 .IP 19247 .nf 19248 \f[C] 19249 c matches character c (c != \[rs], -, ]) 19250 \[rs]c matches reserved character c (c = \[rs], -, ]) 19251 lo - hi matches character c for lo <= c <= hi 19252 \f[R] 19253 .fi 19254 .PP 19255 pattern-list: 19256 .IP 19257 .nf 19258 \f[C] 19259 pattern { , pattern } 19260 comma-separated (without spaces) patterns 19261 \f[R] 19262 .fi 19263 .PP 19264 character classes (see Go regular expression 19265 reference (https://golang.org/pkg/regexp/syntax/)) include: 19266 .IP 19267 .nf 19268 \f[C] 19269 Named character classes (e.g. [\[rs]d], [\[ha]\[rs]d], [\[rs]D], [\[ha]\[rs]D]) 19270 Perl character classes (e.g. \[rs]s, \[rs]S, \[rs]w, \[rs]W) 19271 ASCII character classes (e.g. [[:alnum:]], [[:alpha:]], [[:punct:]], [[:xdigit:]]) 19272 \f[R] 19273 .fi 19274 .PP 19275 regexp for advanced users to insert a regular expression - see below for 19276 more info: 19277 .IP 19278 .nf 19279 \f[C] 19280 Any re2 regular expression not containing \[ga]}}\[ga] 19281 \f[R] 19282 .fi 19283 .PP 19284 If the filter pattern starts with a \f[C]/\f[R] then it only matches at 19285 the top level of the directory tree, \f[B]relative to the root of the 19286 remote\f[R] (not necessarily the root of the drive). 19287 If it does not start with \f[C]/\f[R] then it is matched starting at the 19288 \f[B]end of the path/file name\f[R] but it only matches a complete path 19289 element - it must match from a \f[C]/\f[R] separator or the beginning of 19290 the path/file. 19291 .IP 19292 .nf 19293 \f[C] 19294 file.jpg - matches \[dq]file.jpg\[dq] 19295 - matches \[dq]directory/file.jpg\[dq] 19296 - doesn\[aq]t match \[dq]afile.jpg\[dq] 19297 - doesn\[aq]t match \[dq]directory/afile.jpg\[dq] 19298 /file.jpg - matches \[dq]file.jpg\[dq] in the root directory of the remote 19299 - doesn\[aq]t match \[dq]afile.jpg\[dq] 19300 - doesn\[aq]t match \[dq]directory/file.jpg\[dq] 19301 \f[R] 19302 .fi 19303 .PP 19304 The top level of the remote might not be the top level of the drive. 19305 .PP 19306 E.g. 19307 for a Microsoft Windows local directory structure 19308 .IP 19309 .nf 19310 \f[C] 19311 F: 19312 \[u251C]\[u2500]\[u2500] bkp 19313 \[u251C]\[u2500]\[u2500] data 19314 \[br] \[u251C]\[u2500]\[u2500] excl 19315 \[br] \[br] \[u251C]\[u2500]\[u2500] 123.jpg 19316 \[br] \[br] \[u2514]\[u2500]\[u2500] 456.jpg 19317 \[br] \[u251C]\[u2500]\[u2500] incl 19318 \[br] \[br] \[u2514]\[u2500]\[u2500] document.pdf 19319 \f[R] 19320 .fi 19321 .PP 19322 To copy the contents of folder \f[C]data\f[R] into folder \f[C]bkp\f[R] 19323 excluding the contents of subfolder \f[C]excl\f[R]the following command 19324 treats \f[C]F:\[rs]data\f[R] and \f[C]F:\[rs]bkp\f[R] as top level for 19325 filtering. 19326 .PP 19327 \f[C]rclone copy F:\[rs]data\[rs] F:\[rs]bkp\[rs] --exclude=/excl/**\f[R] 19328 .PP 19329 \f[B]Important\f[R] Use \f[C]/\f[R] in path/file name patterns and not 19330 \f[C]\[rs]\f[R] even if running on Microsoft Windows. 19331 .PP 19332 Simple patterns are case sensitive unless the \f[C]--ignore-case\f[R] 19333 flag is used. 19334 .PP 19335 Without \f[C]--ignore-case\f[R] (default) 19336 .IP 19337 .nf 19338 \f[C] 19339 potato - matches \[dq]potato\[dq] 19340 - doesn\[aq]t match \[dq]POTATO\[dq] 19341 \f[R] 19342 .fi 19343 .PP 19344 With \f[C]--ignore-case\f[R] 19345 .IP 19346 .nf 19347 \f[C] 19348 potato - matches \[dq]potato\[dq] 19349 - matches \[dq]POTATO\[dq] 19350 \f[R] 19351 .fi 19352 .SS Using regular expressions in filter patterns 19353 .PP 19354 The syntax of filter patterns is glob style matching (like 19355 \f[C]bash\f[R] uses) to make things easy for users. 19356 However this does not provide absolute control over the matching, so for 19357 advanced users rclone also provides a regular expression syntax. 19358 .PP 19359 The regular expressions used are as defined in the Go regular expression 19360 reference (https://golang.org/pkg/regexp/syntax/). 19361 Regular expressions should be enclosed in \f[C]{{\f[R] \f[C]}}\f[R]. 19362 They will match only the last path segment if the glob doesn\[aq]t start 19363 with \f[C]/\f[R] or the whole path name if it does. 19364 Note that rclone does not attempt to parse the supplied regular 19365 expression, meaning that using any regular expression filter will 19366 prevent rclone from using directory filter rules, as it will instead 19367 check every path against the supplied regular expression(s). 19368 .PP 19369 Here is how the \f[C]{{regexp}}\f[R] is transformed into an full regular 19370 expression to match the entire path: 19371 .IP 19372 .nf 19373 \f[C] 19374 {{regexp}} becomes (\[ha]|/)(regexp)$ 19375 /{{regexp}} becomes \[ha](regexp)$ 19376 \f[R] 19377 .fi 19378 .PP 19379 Regexp syntax can be mixed with glob syntax, for example 19380 .IP 19381 .nf 19382 \f[C] 19383 *.{{jpe?g}} to match file.jpg, file.jpeg but not file.png 19384 \f[R] 19385 .fi 19386 .PP 19387 You can also use regexp flags - to set case insensitive, for example 19388 .IP 19389 .nf 19390 \f[C] 19391 *.{{(?i)jpg}} to match file.jpg, file.JPG but not file.png 19392 \f[R] 19393 .fi 19394 .PP 19395 Be careful with wildcards in regular expressions - you don\[aq]t want 19396 them to match path separators normally. 19397 To match any file name starting with \f[C]start\f[R] and ending with 19398 \f[C]end\f[R] write 19399 .IP 19400 .nf 19401 \f[C] 19402 {{start[\[ha]/]*end\[rs].jpg}} 19403 \f[R] 19404 .fi 19405 .PP 19406 Not 19407 .IP 19408 .nf 19409 \f[C] 19410 {{start.*end\[rs].jpg}} 19411 \f[R] 19412 .fi 19413 .PP 19414 Which will match a directory called \f[C]start\f[R] with a file called 19415 \f[C]end.jpg\f[R] in it as the \f[C].*\f[R] will match \f[C]/\f[R] 19416 characters. 19417 .PP 19418 Note that you can use \f[C]-vv --dump filters\f[R] to show the filter 19419 patterns in regexp format - rclone implements the glob patterns by 19420 transforming them into regular expressions. 19421 .SS Filter pattern examples 19422 .PP 19423 .TS 19424 tab(@); 19425 l l l l. 19426 T{ 19427 Description 19428 T}@T{ 19429 Pattern 19430 T}@T{ 19431 Matches 19432 T}@T{ 19433 Does not match 19434 T} 19435 _ 19436 T{ 19437 Wildcard 19438 T}@T{ 19439 \f[C]*.jpg\f[R] 19440 T}@T{ 19441 \f[C]/file.jpg\f[R] 19442 T}@T{ 19443 \f[C]/file.png\f[R] 19444 T} 19445 T{ 19446 T}@T{ 19447 T}@T{ 19448 \f[C]/dir/file.jpg\f[R] 19449 T}@T{ 19450 \f[C]/dir/file.png\f[R] 19451 T} 19452 T{ 19453 Rooted 19454 T}@T{ 19455 \f[C]/*.jpg\f[R] 19456 T}@T{ 19457 \f[C]/file.jpg\f[R] 19458 T}@T{ 19459 \f[C]/file.png\f[R] 19460 T} 19461 T{ 19462 T}@T{ 19463 T}@T{ 19464 \f[C]/file2.jpg\f[R] 19465 T}@T{ 19466 \f[C]/dir/file.jpg\f[R] 19467 T} 19468 T{ 19469 Alternates 19470 T}@T{ 19471 \f[C]*.{jpg,png}\f[R] 19472 T}@T{ 19473 \f[C]/file.jpg\f[R] 19474 T}@T{ 19475 \f[C]/file.gif\f[R] 19476 T} 19477 T{ 19478 T}@T{ 19479 T}@T{ 19480 \f[C]/dir/file.png\f[R] 19481 T}@T{ 19482 \f[C]/dir/file.gif\f[R] 19483 T} 19484 T{ 19485 Path Wildcard 19486 T}@T{ 19487 \f[C]dir/**\f[R] 19488 T}@T{ 19489 \f[C]/dir/anyfile\f[R] 19490 T}@T{ 19491 \f[C]file.png\f[R] 19492 T} 19493 T{ 19494 T}@T{ 19495 T}@T{ 19496 \f[C]/subdir/dir/subsubdir/anyfile\f[R] 19497 T}@T{ 19498 \f[C]/subdir/file.png\f[R] 19499 T} 19500 T{ 19501 Any Char 19502 T}@T{ 19503 \f[C]*.t?t\f[R] 19504 T}@T{ 19505 \f[C]/file.txt\f[R] 19506 T}@T{ 19507 \f[C]/file.qxt\f[R] 19508 T} 19509 T{ 19510 T}@T{ 19511 T}@T{ 19512 \f[C]/dir/file.tzt\f[R] 19513 T}@T{ 19514 \f[C]/dir/file.png\f[R] 19515 T} 19516 T{ 19517 Range 19518 T}@T{ 19519 \f[C]*.[a-z]\f[R] 19520 T}@T{ 19521 \f[C]/file.a\f[R] 19522 T}@T{ 19523 \f[C]/file.0\f[R] 19524 T} 19525 T{ 19526 T}@T{ 19527 T}@T{ 19528 \f[C]/dir/file.b\f[R] 19529 T}@T{ 19530 \f[C]/dir/file.1\f[R] 19531 T} 19532 T{ 19533 Escape 19534 T}@T{ 19535 \f[C]*.\[rs]?\[rs]?\[rs]?\f[R] 19536 T}@T{ 19537 \f[C]/file.???\f[R] 19538 T}@T{ 19539 \f[C]/file.abc\f[R] 19540 T} 19541 T{ 19542 T}@T{ 19543 T}@T{ 19544 \f[C]/dir/file.???\f[R] 19545 T}@T{ 19546 \f[C]/dir/file.def\f[R] 19547 T} 19548 T{ 19549 Class 19550 T}@T{ 19551 \f[C]*.\[rs]d\[rs]d\[rs]d\f[R] 19552 T}@T{ 19553 \f[C]/file.012\f[R] 19554 T}@T{ 19555 \f[C]/file.abc\f[R] 19556 T} 19557 T{ 19558 T}@T{ 19559 T}@T{ 19560 \f[C]/dir/file.345\f[R] 19561 T}@T{ 19562 \f[C]/dir/file.def\f[R] 19563 T} 19564 T{ 19565 Regexp 19566 T}@T{ 19567 \f[C]*.{{jpe?g}}\f[R] 19568 T}@T{ 19569 \f[C]/file.jpeg\f[R] 19570 T}@T{ 19571 \f[C]/file.png\f[R] 19572 T} 19573 T{ 19574 T}@T{ 19575 T}@T{ 19576 \f[C]/dir/file.jpg\f[R] 19577 T}@T{ 19578 \f[C]/dir/file.jpeeg\f[R] 19579 T} 19580 T{ 19581 Rooted Regexp 19582 T}@T{ 19583 \f[C]/{{.*\[rs].jpe?g}}\f[R] 19584 T}@T{ 19585 \f[C]/file.jpeg\f[R] 19586 T}@T{ 19587 \f[C]/file.png\f[R] 19588 T} 19589 T{ 19590 T}@T{ 19591 T}@T{ 19592 \f[C]/file.jpg\f[R] 19593 T}@T{ 19594 \f[C]/dir/file.jpg\f[R] 19595 T} 19596 .TE 19597 .SS How filter rules are applied to files 19598 .PP 19599 Rclone path/file name filters are made up of one or more of the 19600 following flags: 19601 .IP \[bu] 2 19602 \f[C]--include\f[R] 19603 .IP \[bu] 2 19604 \f[C]--include-from\f[R] 19605 .IP \[bu] 2 19606 \f[C]--exclude\f[R] 19607 .IP \[bu] 2 19608 \f[C]--exclude-from\f[R] 19609 .IP \[bu] 2 19610 \f[C]--filter\f[R] 19611 .IP \[bu] 2 19612 \f[C]--filter-from\f[R] 19613 .PP 19614 There can be more than one instance of individual flags. 19615 .PP 19616 Rclone internally uses a combined list of all the include and exclude 19617 rules. 19618 The order in which rules are processed can influence the result of the 19619 filter. 19620 .PP 19621 All flags of the same type are processed together in the order above, 19622 regardless of what order the different types of flags are included on 19623 the command line. 19624 .PP 19625 Multiple instances of the same flag are processed from left to right 19626 according to their position in the command line. 19627 .PP 19628 To mix up the order of processing includes and excludes use 19629 \f[C]--filter...\f[R] flags. 19630 .PP 19631 Within \f[C]--include-from\f[R], \f[C]--exclude-from\f[R] and 19632 \f[C]--filter-from\f[R] flags rules are processed from top to bottom of 19633 the referenced file. 19634 .PP 19635 If there is an \f[C]--include\f[R] or \f[C]--include-from\f[R] flag 19636 specified, rclone implies a \f[C]- **\f[R] rule which it adds to the 19637 bottom of the internal rule list. 19638 Specifying a \f[C]+\f[R] rule with a \f[C]--filter...\f[R] flag does not 19639 imply that rule. 19640 .PP 19641 Each path/file name passed through rclone is matched against the 19642 combined filter list. 19643 At first match to a rule the path/file name is included or excluded and 19644 no further filter rules are processed for that path/file. 19645 .PP 19646 If rclone does not find a match, after testing against all rules 19647 (including the implied rule if appropriate), the path/file name is 19648 included. 19649 .PP 19650 Any path/file included at that stage is processed by the rclone command. 19651 .PP 19652 \f[C]--files-from\f[R] and \f[C]--files-from-raw\f[R] flags over-ride 19653 and cannot be combined with other filter options. 19654 .PP 19655 To see the internal combined rule list, in regular expression form, for 19656 a command add the \f[C]--dump filters\f[R] flag. 19657 Running an rclone command with \f[C]--dump filters\f[R] and 19658 \f[C]-vv\f[R] flags lists the internal filter elements and shows how 19659 they are applied to each source path/file. 19660 There is not currently a means provided to pass regular expression 19661 filter options into rclone directly though character class filter rules 19662 contain character classes. 19663 Go regular expression reference (https://golang.org/pkg/regexp/syntax/) 19664 .SS How filter rules are applied to directories 19665 .PP 19666 Rclone commands are applied to path/file names not directories. 19667 The entire contents of a directory can be matched to a filter by the 19668 pattern \f[C]directory/*\f[R] or recursively by \f[C]directory/**\f[R]. 19669 .PP 19670 Directory filter rules are defined with a closing \f[C]/\f[R] separator. 19671 .PP 19672 E.g. 19673 \f[C]/directory/subdirectory/\f[R] is an rclone directory filter rule. 19674 .PP 19675 Rclone commands can use directory filter rules to determine whether they 19676 recurse into subdirectories. 19677 This potentially optimises access to a remote by avoiding listing 19678 unnecessary directories. 19679 Whether optimisation is desirable depends on the specific filter rules 19680 and source remote content. 19681 .PP 19682 If any regular expression filters are in use, then no directory 19683 recursion optimisation is possible, as rclone must check every path 19684 against the supplied regular expression(s). 19685 .PP 19686 Directory recursion optimisation occurs if either: 19687 .IP \[bu] 2 19688 A source remote does not support the rclone \f[C]ListR\f[R] primitive. 19689 local, sftp, Microsoft OneDrive and WebDAV do not support 19690 \f[C]ListR\f[R]. 19691 Google Drive and most bucket type storage do. 19692 Full list (https://rclone.org/overview/#optional-features) 19693 .IP \[bu] 2 19694 On other remotes (those that support \f[C]ListR\f[R]), if the rclone 19695 command is not naturally recursive, and provided it is not run with the 19696 \f[C]--fast-list\f[R] flag. 19697 \f[C]ls\f[R], \f[C]lsf -R\f[R] and \f[C]size\f[R] are naturally 19698 recursive but \f[C]sync\f[R], \f[C]copy\f[R] and \f[C]move\f[R] are not. 19699 .IP \[bu] 2 19700 Whenever the \f[C]--disable ListR\f[R] flag is applied to an rclone 19701 command. 19702 .PP 19703 Rclone commands imply directory filter rules from path/file filter 19704 rules. 19705 To view the directory filter rules rclone has implied for a command 19706 specify the \f[C]--dump filters\f[R] flag. 19707 .PP 19708 E.g. 19709 for an include rule 19710 .IP 19711 .nf 19712 \f[C] 19713 /a/*.jpg 19714 \f[R] 19715 .fi 19716 .PP 19717 Rclone implies the directory include rule 19718 .IP 19719 .nf 19720 \f[C] 19721 /a/ 19722 \f[R] 19723 .fi 19724 .PP 19725 Directory filter rules specified in an rclone command can limit the 19726 scope of an rclone command but path/file filters still have to be 19727 specified. 19728 .PP 19729 E.g. 19730 \f[C]rclone ls remote: --include /directory/\f[R] will not match any 19731 files. 19732 Because it is an \f[C]--include\f[R] option the \f[C]--exclude **\f[R] 19733 rule is implied, and the \f[C]/directory/\f[R] pattern serves only to 19734 optimise access to the remote by ignoring everything outside of that 19735 directory. 19736 .PP 19737 E.g. 19738 \f[C]rclone ls remote: --filter-from filter-list.txt\f[R] with a file 19739 \f[C]filter-list.txt\f[R]: 19740 .IP 19741 .nf 19742 \f[C] 19743 - /dir1/ 19744 - /dir2/ 19745 + *.pdf 19746 - ** 19747 \f[R] 19748 .fi 19749 .PP 19750 All files in directories \f[C]dir1\f[R] or \f[C]dir2\f[R] or their 19751 subdirectories are completely excluded from the listing. 19752 Only files of suffix \f[C]pdf\f[R] in the root of \f[C]remote:\f[R] or 19753 its subdirectories are listed. 19754 The \f[C]- **\f[R] rule prevents listing of any path/files not 19755 previously matched by the rules above. 19756 .PP 19757 Option \f[C]exclude-if-present\f[R] creates a directory exclude rule 19758 based on the presence of a file in a directory and takes precedence over 19759 other rclone directory filter rules. 19760 .PP 19761 When using pattern list syntax, if a pattern item contains either 19762 \f[C]/\f[R] or \f[C]**\f[R], then rclone will not able to imply a 19763 directory filter rule from this pattern list. 19764 .PP 19765 E.g. 19766 for an include rule 19767 .IP 19768 .nf 19769 \f[C] 19770 {dir1/**,dir2/**} 19771 \f[R] 19772 .fi 19773 .PP 19774 Rclone will match files below directories \f[C]dir1\f[R] or 19775 \f[C]dir2\f[R] only, but will not be able to use this filter to exclude 19776 a directory \f[C]dir3\f[R] from being traversed. 19777 .PP 19778 Directory recursion optimisation may affect performance, but normally 19779 not the result. 19780 One exception to this is sync operations with option 19781 \f[C]--create-empty-src-dirs\f[R], where any traversed empty directories 19782 will be created. 19783 With the pattern list example \f[C]{dir1/**,dir2/**}\f[R] above, this 19784 would create an empty directory \f[C]dir3\f[R] on destination (when it 19785 exists on source). 19786 Changing the filter to \f[C]{dir1,dir2}/**\f[R], or splitting it into 19787 two include rules \f[C]--include dir1/** --include dir2/**\f[R], will 19788 match the same files while also filtering directories, with the result 19789 that an empty directory \f[C]dir3\f[R] will no longer be created. 19790 .SS \f[C]--exclude\f[R] - Exclude files matching pattern 19791 .PP 19792 Excludes path/file names from an rclone command based on a single 19793 exclude rule. 19794 .PP 19795 This flag can be repeated. 19796 See above for the order filter flags are processed in. 19797 .PP 19798 \f[C]--exclude\f[R] should not be used with \f[C]--include\f[R], 19799 \f[C]--include-from\f[R], \f[C]--filter\f[R] or \f[C]--filter-from\f[R] 19800 flags. 19801 .PP 19802 \f[C]--exclude\f[R] has no effect when combined with 19803 \f[C]--files-from\f[R] or \f[C]--files-from-raw\f[R] flags. 19804 .PP 19805 E.g. 19806 \f[C]rclone ls remote: --exclude *.bak\f[R] excludes all .bak files from 19807 listing. 19808 .PP 19809 E.g. 19810 \f[C]rclone size remote: \[dq]--exclude /dir/**\[dq]\f[R] returns the 19811 total size of all files on \f[C]remote:\f[R] excluding those in root 19812 directory \f[C]dir\f[R] and sub directories. 19813 .PP 19814 E.g. 19815 on Microsoft Windows 19816 \f[C]rclone ls remote: --exclude \[dq]*\[rs][{JP,KR,HK}\[rs]]*\[dq]\f[R] 19817 lists the files in \f[C]remote:\f[R] without \f[C][JP]\f[R] or 19818 \f[C][KR]\f[R] or \f[C][HK]\f[R] in their name. 19819 Quotes prevent the shell from interpreting the \f[C]\[rs]\f[R] 19820 characters.\f[C]\[rs]\f[R] characters escape the \f[C][\f[R] and 19821 \f[C]]\f[R] so an rclone filter treats them literally rather than as a 19822 character-range. 19823 The \f[C]{\f[R] and \f[C]}\f[R] define an rclone pattern list. 19824 For other operating systems single quotes are required ie 19825 \f[C]rclone ls remote: --exclude \[aq]*\[rs][{JP,KR,HK}\[rs]]*\[aq]\f[R] 19826 .SS \f[C]--exclude-from\f[R] - Read exclude patterns from file 19827 .PP 19828 Excludes path/file names from an rclone command based on rules in a 19829 named file. 19830 The file contains a list of remarks and pattern rules. 19831 .PP 19832 For an example \f[C]exclude-file.txt\f[R]: 19833 .IP 19834 .nf 19835 \f[C] 19836 # a sample exclude rule file 19837 *.bak 19838 file2.jpg 19839 \f[R] 19840 .fi 19841 .PP 19842 \f[C]rclone ls remote: --exclude-from exclude-file.txt\f[R] lists the 19843 files on \f[C]remote:\f[R] except those named \f[C]file2.jpg\f[R] or 19844 with a suffix \f[C].bak\f[R]. 19845 That is equivalent to 19846 \f[C]rclone ls remote: --exclude file2.jpg --exclude \[dq]*.bak\[dq]\f[R]. 19847 .PP 19848 This flag can be repeated. 19849 See above for the order filter flags are processed in. 19850 .PP 19851 The \f[C]--exclude-from\f[R] flag is useful where multiple exclude 19852 filter rules are applied to an rclone command. 19853 .PP 19854 \f[C]--exclude-from\f[R] should not be used with \f[C]--include\f[R], 19855 \f[C]--include-from\f[R], \f[C]--filter\f[R] or \f[C]--filter-from\f[R] 19856 flags. 19857 .PP 19858 \f[C]--exclude-from\f[R] has no effect when combined with 19859 \f[C]--files-from\f[R] or \f[C]--files-from-raw\f[R] flags. 19860 .PP 19861 \f[C]--exclude-from\f[R] followed by \f[C]-\f[R] reads filter rules from 19862 standard input. 19863 .SS \f[C]--include\f[R] - Include files matching pattern 19864 .PP 19865 Adds a single include rule based on path/file names to an rclone 19866 command. 19867 .PP 19868 This flag can be repeated. 19869 See above for the order filter flags are processed in. 19870 .PP 19871 \f[C]--include\f[R] has no effect when combined with 19872 \f[C]--files-from\f[R] or \f[C]--files-from-raw\f[R] flags. 19873 .PP 19874 \f[C]--include\f[R] implies \f[C]--exclude **\f[R] at the end of an 19875 rclone internal filter list. 19876 Therefore if you mix \f[C]--include\f[R] and \f[C]--include-from\f[R] 19877 flags with \f[C]--exclude\f[R], \f[C]--exclude-from\f[R], 19878 \f[C]--filter\f[R] or \f[C]--filter-from\f[R], you must use include 19879 rules for all the files you want in the include statement. 19880 For more flexibility use the \f[C]--filter-from\f[R] flag. 19881 .PP 19882 E.g. 19883 \f[C]rclone ls remote: --include \[dq]*.{png,jpg}\[dq]\f[R] lists the 19884 files on \f[C]remote:\f[R] with suffix \f[C].png\f[R] and 19885 \f[C].jpg\f[R]. 19886 All other files are excluded. 19887 .PP 19888 E.g. 19889 multiple rclone copy commands can be combined with \f[C]--include\f[R] 19890 and a pattern-list. 19891 .IP 19892 .nf 19893 \f[C] 19894 rclone copy /vol1/A remote:A 19895 rclone copy /vol1/B remote:B 19896 \f[R] 19897 .fi 19898 .PP 19899 is equivalent to: 19900 .IP 19901 .nf 19902 \f[C] 19903 rclone copy /vol1 remote: --include \[dq]{A,B}/**\[dq] 19904 \f[R] 19905 .fi 19906 .PP 19907 E.g. 19908 \f[C]rclone ls remote:/wheat --include \[dq]??[\[ha][:punct:]]*\[dq]\f[R] 19909 lists the files \f[C]remote:\f[R] directory \f[C]wheat\f[R] (and 19910 subdirectories) whose third character is not punctuation. 19911 This example uses an ASCII character 19912 class (https://golang.org/pkg/regexp/syntax/). 19913 .SS \f[C]--include-from\f[R] - Read include patterns from file 19914 .PP 19915 Adds path/file names to an rclone command based on rules in a named 19916 file. 19917 The file contains a list of remarks and pattern rules. 19918 .PP 19919 For an example \f[C]include-file.txt\f[R]: 19920 .IP 19921 .nf 19922 \f[C] 19923 # a sample include rule file 19924 *.jpg 19925 file2.avi 19926 \f[R] 19927 .fi 19928 .PP 19929 \f[C]rclone ls remote: --include-from include-file.txt\f[R] lists the 19930 files on \f[C]remote:\f[R] with name \f[C]file2.avi\f[R] or suffix 19931 \f[C].jpg\f[R]. 19932 That is equivalent to 19933 \f[C]rclone ls remote: --include file2.avi --include \[dq]*.jpg\[dq]\f[R]. 19934 .PP 19935 This flag can be repeated. 19936 See above for the order filter flags are processed in. 19937 .PP 19938 The \f[C]--include-from\f[R] flag is useful where multiple include 19939 filter rules are applied to an rclone command. 19940 .PP 19941 \f[C]--include-from\f[R] implies \f[C]--exclude **\f[R] at the end of an 19942 rclone internal filter list. 19943 Therefore if you mix \f[C]--include\f[R] and \f[C]--include-from\f[R] 19944 flags with \f[C]--exclude\f[R], \f[C]--exclude-from\f[R], 19945 \f[C]--filter\f[R] or \f[C]--filter-from\f[R], you must use include 19946 rules for all the files you want in the include statement. 19947 For more flexibility use the \f[C]--filter-from\f[R] flag. 19948 .PP 19949 \f[C]--exclude-from\f[R] has no effect when combined with 19950 \f[C]--files-from\f[R] or \f[C]--files-from-raw\f[R] flags. 19951 .PP 19952 \f[C]--exclude-from\f[R] followed by \f[C]-\f[R] reads filter rules from 19953 standard input. 19954 .SS \f[C]--filter\f[R] - Add a file-filtering rule 19955 .PP 19956 Specifies path/file names to an rclone command, based on a single 19957 include or exclude rule, in \f[C]+\f[R] or \f[C]-\f[R] format. 19958 .PP 19959 This flag can be repeated. 19960 See above for the order filter flags are processed in. 19961 .PP 19962 \f[C]--filter +\f[R] differs from \f[C]--include\f[R]. 19963 In the case of \f[C]--include\f[R] rclone implies an 19964 \f[C]--exclude *\f[R] rule which it adds to the bottom of the internal 19965 rule list. 19966 \f[C]--filter...+\f[R] does not imply that rule. 19967 .PP 19968 \f[C]--filter\f[R] has no effect when combined with 19969 \f[C]--files-from\f[R] or \f[C]--files-from-raw\f[R] flags. 19970 .PP 19971 \f[C]--filter\f[R] should not be used with \f[C]--include\f[R], 19972 \f[C]--include-from\f[R], \f[C]--exclude\f[R] or 19973 \f[C]--exclude-from\f[R] flags. 19974 .PP 19975 E.g. 19976 \f[C]rclone ls remote: --filter \[dq]- *.bak\[dq]\f[R] excludes all 19977 \f[C].bak\f[R] files from a list of \f[C]remote:\f[R]. 19978 .SS \f[C]--filter-from\f[R] - Read filtering patterns from a file 19979 .PP 19980 Adds path/file names to an rclone command based on rules in a named 19981 file. 19982 The file contains a list of remarks and pattern rules. 19983 Include rules start with \f[C]+\f[R] and exclude rules with \f[C]-\f[R]. 19984 \f[C]!\f[R] clears existing rules. 19985 Rules are processed in the order they are defined. 19986 .PP 19987 This flag can be repeated. 19988 See above for the order filter flags are processed in. 19989 .PP 19990 Arrange the order of filter rules with the most restrictive first and 19991 work down. 19992 .PP 19993 E.g. 19994 for \f[C]filter-file.txt\f[R]: 19995 .IP 19996 .nf 19997 \f[C] 19998 # a sample filter rule file 19999 - secret*.jpg 20000 + *.jpg 20001 + *.png 20002 + file2.avi 20003 - /dir/Trash/** 20004 + /dir/** 20005 # exclude everything else 20006 - * 20007 \f[R] 20008 .fi 20009 .PP 20010 \f[C]rclone ls remote: --filter-from filter-file.txt\f[R] lists the 20011 path/files on \f[C]remote:\f[R] including all \f[C]jpg\f[R] and 20012 \f[C]png\f[R] files, excluding any matching \f[C]secret*.jpg\f[R] and 20013 including \f[C]file2.avi\f[R]. 20014 It also includes everything in the directory \f[C]dir\f[R] at the root 20015 of \f[C]remote\f[R], except \f[C]remote:dir/Trash\f[R] which it 20016 excludes. 20017 Everything else is excluded. 20018 .PP 20019 E.g. 20020 for an alternative \f[C]filter-file.txt\f[R]: 20021 .IP 20022 .nf 20023 \f[C] 20024 - secret*.jpg 20025 + *.jpg 20026 + *.png 20027 + file2.avi 20028 - * 20029 \f[R] 20030 .fi 20031 .PP 20032 Files \f[C]file1.jpg\f[R], \f[C]file3.png\f[R] and \f[C]file2.avi\f[R] 20033 are listed whilst \f[C]secret17.jpg\f[R] and files without the suffix 20034 \f[C].jpg\f[R] or \f[C].png\f[R] are excluded. 20035 .PP 20036 E.g. 20037 for an alternative \f[C]filter-file.txt\f[R]: 20038 .IP 20039 .nf 20040 \f[C] 20041 + *.jpg 20042 + *.gif 20043 ! 20044 + 42.doc 20045 - * 20046 \f[R] 20047 .fi 20048 .PP 20049 Only file 42.doc is listed. 20050 Prior rules are cleared by the \f[C]!\f[R]. 20051 .SS \f[C]--files-from\f[R] - Read list of source-file names 20052 .PP 20053 Adds path/files to an rclone command from a list in a named file. 20054 Rclone processes the path/file names in the order of the list, and no 20055 others. 20056 .PP 20057 Other filter flags (\f[C]--include\f[R], \f[C]--include-from\f[R], 20058 \f[C]--exclude\f[R], \f[C]--exclude-from\f[R], \f[C]--filter\f[R] and 20059 \f[C]--filter-from\f[R]) are ignored when \f[C]--files-from\f[R] is 20060 used. 20061 .PP 20062 \f[C]--files-from\f[R] expects a list of files as its input. 20063 Leading or trailing whitespace is stripped from the input lines. 20064 Lines starting with \f[C]#\f[R] or \f[C];\f[R] are ignored. 20065 .PP 20066 Rclone commands with a \f[C]--files-from\f[R] flag traverse the remote, 20067 treating the names in \f[C]--files-from\f[R] as a set of filters. 20068 .PP 20069 If the \f[C]--no-traverse\f[R] and \f[C]--files-from\f[R] flags are used 20070 together an rclone command does not traverse the remote. 20071 Instead it addresses each path/file named in the file individually. 20072 For each path/file name, that requires typically 1 API call. 20073 This can be efficient for a short \f[C]--files-from\f[R] list and a 20074 remote containing many files. 20075 .PP 20076 Rclone commands do not error if any names in the \f[C]--files-from\f[R] 20077 file are missing from the source remote. 20078 .PP 20079 The \f[C]--files-from\f[R] flag can be repeated in a single rclone 20080 command to read path/file names from more than one file. 20081 The files are read from left to right along the command line. 20082 .PP 20083 Paths within the \f[C]--files-from\f[R] file are interpreted as starting 20084 with the root specified in the rclone command. 20085 Leading \f[C]/\f[R] separators are ignored. 20086 See --files-from-raw if you need the input to be processed in a raw 20087 manner. 20088 .PP 20089 E.g. 20090 for a file \f[C]files-from.txt\f[R]: 20091 .IP 20092 .nf 20093 \f[C] 20094 # comment 20095 file1.jpg 20096 subdir/file2.jpg 20097 \f[R] 20098 .fi 20099 .PP 20100 \f[C]rclone copy --files-from files-from.txt /home/me/pics remote:pics\f[R] 20101 copies the following, if they exist, and only those files. 20102 .IP 20103 .nf 20104 \f[C] 20105 /home/me/pics/file1.jpg \[->] remote:pics/file1.jpg 20106 /home/me/pics/subdir/file2.jpg \[->] remote:pics/subdir/file2.jpg 20107 \f[R] 20108 .fi 20109 .PP 20110 E.g. 20111 to copy the following files referenced by their absolute paths: 20112 .IP 20113 .nf 20114 \f[C] 20115 /home/user1/42 20116 /home/user1/dir/ford 20117 /home/user2/prefect 20118 \f[R] 20119 .fi 20120 .PP 20121 First find a common subdirectory - in this case \f[C]/home\f[R] and put 20122 the remaining files in \f[C]files-from.txt\f[R] with or without leading 20123 \f[C]/\f[R], e.g. 20124 .IP 20125 .nf 20126 \f[C] 20127 user1/42 20128 user1/dir/ford 20129 user2/prefect 20130 \f[R] 20131 .fi 20132 .PP 20133 Then copy these to a remote: 20134 .IP 20135 .nf 20136 \f[C] 20137 rclone copy --files-from files-from.txt /home remote:backup 20138 \f[R] 20139 .fi 20140 .PP 20141 The three files are transferred as follows: 20142 .IP 20143 .nf 20144 \f[C] 20145 /home/user1/42 \[->] remote:backup/user1/important 20146 /home/user1/dir/ford \[->] remote:backup/user1/dir/file 20147 /home/user2/prefect \[->] remote:backup/user2/stuff 20148 \f[R] 20149 .fi 20150 .PP 20151 Alternatively if \f[C]/\f[R] is chosen as root \f[C]files-from.txt\f[R] 20152 will be: 20153 .IP 20154 .nf 20155 \f[C] 20156 /home/user1/42 20157 /home/user1/dir/ford 20158 /home/user2/prefect 20159 \f[R] 20160 .fi 20161 .PP 20162 The copy command will be: 20163 .IP 20164 .nf 20165 \f[C] 20166 rclone copy --files-from files-from.txt / remote:backup 20167 \f[R] 20168 .fi 20169 .PP 20170 Then there will be an extra \f[C]home\f[R] directory on the remote: 20171 .IP 20172 .nf 20173 \f[C] 20174 /home/user1/42 \[->] remote:backup/home/user1/42 20175 /home/user1/dir/ford \[->] remote:backup/home/user1/dir/ford 20176 /home/user2/prefect \[->] remote:backup/home/user2/prefect 20177 \f[R] 20178 .fi 20179 .SS \f[C]--files-from-raw\f[R] - Read list of source-file names without any processing 20180 .PP 20181 This flag is the same as \f[C]--files-from\f[R] except that input is 20182 read in a raw manner. 20183 Lines with leading / trailing whitespace, and lines starting with 20184 \f[C];\f[R] or \f[C]#\f[R] are read without any processing. 20185 rclone lsf (https://rclone.org/commands/rclone_lsf/) has a compatible 20186 format that can be used to export file lists from remotes for input to 20187 \f[C]--files-from-raw\f[R]. 20188 .SS \f[C]--ignore-case\f[R] - make searches case insensitive 20189 .PP 20190 By default, rclone filter patterns are case sensitive. 20191 The \f[C]--ignore-case\f[R] flag makes all of the filters patterns on 20192 the command line case insensitive. 20193 .PP 20194 E.g. 20195 \f[C]--include \[dq]zaphod.txt\[dq]\f[R] does not match a file 20196 \f[C]Zaphod.txt\f[R]. 20197 With \f[C]--ignore-case\f[R] a match is made. 20198 .SS Quoting shell metacharacters 20199 .PP 20200 Rclone commands with filter patterns containing shell metacharacters may 20201 not as work as expected in your shell and may require quoting. 20202 .PP 20203 E.g. 20204 linux, OSX (\f[C]*\f[R] metacharacter) 20205 .IP \[bu] 2 20206 \f[C]--include \[rs]*.jpg\f[R] 20207 .IP \[bu] 2 20208 \f[C]--include \[aq]*.jpg\[aq]\f[R] 20209 .IP \[bu] 2 20210 \f[C]--include=\[aq]*.jpg\[aq]\f[R] 20211 .PP 20212 Microsoft Windows expansion is done by the command, not shell, so 20213 \f[C]--include *.jpg\f[R] does not require quoting. 20214 .PP 20215 If the rclone error 20216 \f[C]Command .... needs .... arguments maximum: you provided .... non flag arguments:\f[R] 20217 is encountered, the cause is commonly spaces within the name of a remote 20218 or flag value. 20219 The fix then is to quote values containing spaces. 20220 .SS Other filters 20221 .SS \f[C]--min-size\f[R] - Don\[aq]t transfer any file smaller than this 20222 .PP 20223 Controls the minimum size file within the scope of an rclone command. 20224 Default units are \f[C]KiB\f[R] but abbreviations \f[C]K\f[R], 20225 \f[C]M\f[R], \f[C]G\f[R], \f[C]T\f[R] or \f[C]P\f[R] are valid. 20226 .PP 20227 E.g. 20228 \f[C]rclone ls remote: --min-size 50k\f[R] lists files on 20229 \f[C]remote:\f[R] of 50 KiB size or larger. 20230 .PP 20231 See the size option docs (https://rclone.org/docs/#size-option) for more 20232 info. 20233 .SS \f[C]--max-size\f[R] - Don\[aq]t transfer any file larger than this 20234 .PP 20235 Controls the maximum size file within the scope of an rclone command. 20236 Default units are \f[C]KiB\f[R] but abbreviations \f[C]K\f[R], 20237 \f[C]M\f[R], \f[C]G\f[R], \f[C]T\f[R] or \f[C]P\f[R] are valid. 20238 .PP 20239 E.g. 20240 \f[C]rclone ls remote: --max-size 1G\f[R] lists files on 20241 \f[C]remote:\f[R] of 1 GiB size or smaller. 20242 .PP 20243 See the size option docs (https://rclone.org/docs/#size-option) for more 20244 info. 20245 .SS \f[C]--max-age\f[R] - Don\[aq]t transfer any file older than this 20246 .PP 20247 Controls the maximum age of files within the scope of an rclone command. 20248 .PP 20249 \f[C]--max-age\f[R] applies only to files and not to directories. 20250 .PP 20251 E.g. 20252 \f[C]rclone ls remote: --max-age 2d\f[R] lists files on 20253 \f[C]remote:\f[R] of 2 days old or less. 20254 .PP 20255 See the time option docs (https://rclone.org/docs/#time-option) for 20256 valid formats. 20257 .SS \f[C]--min-age\f[R] - Don\[aq]t transfer any file younger than this 20258 .PP 20259 Controls the minimum age of files within the scope of an rclone command. 20260 (see \f[C]--max-age\f[R] for valid formats) 20261 .PP 20262 \f[C]--min-age\f[R] applies only to files and not to directories. 20263 .PP 20264 E.g. 20265 \f[C]rclone ls remote: --min-age 2d\f[R] lists files on 20266 \f[C]remote:\f[R] of 2 days old or more. 20267 .PP 20268 See the time option docs (https://rclone.org/docs/#time-option) for 20269 valid formats. 20270 .SS Other flags 20271 .SS \f[C]--delete-excluded\f[R] - Delete files on dest excluded from sync 20272 .PP 20273 \f[B]Important\f[R] this flag is dangerous to your data - use with 20274 \f[C]--dry-run\f[R] and \f[C]-v\f[R] first. 20275 .PP 20276 In conjunction with \f[C]rclone sync\f[R], \f[C]--delete-excluded\f[R] 20277 deletes any files on the destination which are excluded from the 20278 command. 20279 .PP 20280 E.g. 20281 the scope of \f[C]rclone sync --interactive A: B:\f[R] can be 20282 restricted: 20283 .IP 20284 .nf 20285 \f[C] 20286 rclone --min-size 50k --delete-excluded sync A: B: 20287 \f[R] 20288 .fi 20289 .PP 20290 All files on \f[C]B:\f[R] which are less than 50 KiB are deleted because 20291 they are excluded from the rclone sync command. 20292 .SS \f[C]--dump filters\f[R] - dump the filters to the output 20293 .PP 20294 Dumps the defined filters to standard output in regular expression 20295 format. 20296 .PP 20297 Useful for debugging. 20298 .SS Exclude directory based on a file 20299 .PP 20300 The \f[C]--exclude-if-present\f[R] flag controls whether a directory is 20301 within the scope of an rclone command based on the presence of a named 20302 file within it. 20303 The flag can be repeated to check for multiple file names, presence of 20304 any of them will exclude the directory. 20305 .PP 20306 This flag has a priority over other filter flags. 20307 .PP 20308 E.g. 20309 for the following directory structure: 20310 .IP 20311 .nf 20312 \f[C] 20313 dir1/file1 20314 dir1/dir2/file2 20315 dir1/dir2/dir3/file3 20316 dir1/dir2/dir3/.ignore 20317 \f[R] 20318 .fi 20319 .PP 20320 The command \f[C]rclone ls --exclude-if-present .ignore dir1\f[R] does 20321 not list \f[C]dir3\f[R], \f[C]file3\f[R] or \f[C].ignore\f[R]. 20322 .SS Metadata filters 20323 .PP 20324 The metadata filters work in a very similar way to the normal file name 20325 filters, except they match metadata (https://rclone.org/docs/#metadata) 20326 on the object. 20327 .PP 20328 The metadata should be specified as \f[C]key=value\f[R] patterns. 20329 This may be wildcarded using the normal filter patterns or regular 20330 expressions. 20331 .PP 20332 For example if you wished to list only local files with a mode of 20333 \f[C]100664\f[R] you could do that with: 20334 .IP 20335 .nf 20336 \f[C] 20337 rclone lsf -M --files-only --metadata-include \[dq]mode=100664\[dq] . 20338 \f[R] 20339 .fi 20340 .PP 20341 Or if you wished to show files with an \f[C]atime\f[R], \f[C]mtime\f[R] 20342 or \f[C]btime\f[R] at a given date: 20343 .IP 20344 .nf 20345 \f[C] 20346 rclone lsf -M --files-only --metadata-include \[dq][abm]time=2022-12-16*\[dq] . 20347 \f[R] 20348 .fi 20349 .PP 20350 Like file filtering, metadata filtering only applies to files not to 20351 directories. 20352 .PP 20353 The filters can be applied using these flags. 20354 .IP \[bu] 2 20355 \f[C]--metadata-include\f[R] - Include metadatas matching pattern 20356 .IP \[bu] 2 20357 \f[C]--metadata-include-from\f[R] - Read metadata include patterns from 20358 file (use - to read from stdin) 20359 .IP \[bu] 2 20360 \f[C]--metadata-exclude\f[R] - Exclude metadatas matching pattern 20361 .IP \[bu] 2 20362 \f[C]--metadata-exclude-from\f[R] - Read metadata exclude patterns from 20363 file (use - to read from stdin) 20364 .IP \[bu] 2 20365 \f[C]--metadata-filter\f[R] - Add a metadata filtering rule 20366 .IP \[bu] 2 20367 \f[C]--metadata-filter-from\f[R] - Read metadata filtering patterns from 20368 a file (use - to read from stdin) 20369 .PP 20370 Each flag can be repeated. 20371 See the section on how filter rules are applied for more details - these 20372 flags work in an identical way to the file name filtering flags, but 20373 instead of file name patterns have metadata patterns. 20374 .SS Common pitfalls 20375 .PP 20376 The most frequent filter support issues on the rclone 20377 forum (https://forum.rclone.org/) are: 20378 .IP \[bu] 2 20379 Not using paths relative to the root of the remote 20380 .IP \[bu] 2 20381 Not using \f[C]/\f[R] to match from the root of a remote 20382 .IP \[bu] 2 20383 Not using \f[C]**\f[R] to match the contents of a directory 20384 .SH GUI (Experimental) 20385 .PP 20386 Rclone can serve a web based GUI (graphical user interface). 20387 This is somewhat experimental at the moment so things may be subject to 20388 change. 20389 .PP 20390 Run this command in a terminal and rclone will download and then display 20391 the GUI in a web browser. 20392 .IP 20393 .nf 20394 \f[C] 20395 rclone rcd --rc-web-gui 20396 \f[R] 20397 .fi 20398 .PP 20399 This will produce logs like this and rclone needs to continue to run to 20400 serve the GUI: 20401 .IP 20402 .nf 20403 \f[C] 20404 2019/08/25 11:40:14 NOTICE: A new release for gui is present at https://github.com/artpar/artpar-webui-react/releases/download/v0.0.6/currentbuild.zip 20405 2019/08/25 11:40:14 NOTICE: Downloading webgui binary. Please wait. [Size: 3813937, Path : /home/USER/.cache/rclone/webgui/v0.0.6.zip] 20406 2019/08/25 11:40:16 NOTICE: Unzipping 20407 2019/08/25 11:40:16 NOTICE: Serving remote control on http://127.0.0.1:5572/ 20408 \f[R] 20409 .fi 20410 .PP 20411 This assumes you are running rclone locally on your machine. 20412 It is possible to separate the rclone and the GUI - see below for 20413 details. 20414 .PP 20415 If you wish to check for updates then you can add 20416 \f[C]--rc-web-gui-update\f[R] to the command line. 20417 .PP 20418 If you find your GUI broken, you may force it to update by add 20419 \f[C]--rc-web-gui-force-update\f[R]. 20420 .PP 20421 By default, rclone will open your browser. 20422 Add \f[C]--rc-web-gui-no-open-browser\f[R] to disable this feature. 20423 .SS Using the GUI 20424 .PP 20425 Once the GUI opens, you will be looking at the dashboard which has an 20426 overall overview. 20427 .PP 20428 On the left hand side you will see a series of view buttons you can 20429 click on: 20430 .IP \[bu] 2 20431 Dashboard - main overview 20432 .IP \[bu] 2 20433 Configs - examine and create new configurations 20434 .IP \[bu] 2 20435 Explorer - view, download and upload files to the cloud storage systems 20436 .IP \[bu] 2 20437 Backend - view or alter the backend config 20438 .IP \[bu] 2 20439 Log out 20440 .PP 20441 (More docs and walkthrough video to come!) 20442 .SS How it works 20443 .PP 20444 When you run the \f[C]rclone rcd --rc-web-gui\f[R] this is what happens 20445 .IP \[bu] 2 20446 Rclone starts but only runs the remote control API (\[dq]rc\[dq]). 20447 .IP \[bu] 2 20448 The API is bound to localhost with an auto-generated username and 20449 password. 20450 .IP \[bu] 2 20451 If the API bundle is missing then rclone will download it. 20452 .IP \[bu] 2 20453 rclone will start serving the files from the API bundle over the same 20454 port as the API 20455 .IP \[bu] 2 20456 rclone will open the browser with a \f[C]login_token\f[R] so it can log 20457 straight in. 20458 .SS Advanced use 20459 .PP 20460 The \f[C]rclone rcd\f[R] may use any of the flags documented on the rc 20461 page (https://rclone.org/rc/#supported-parameters). 20462 .PP 20463 The flag \f[C]--rc-web-gui\f[R] is shorthand for 20464 .IP \[bu] 2 20465 Download the web GUI if necessary 20466 .IP \[bu] 2 20467 Check we are using some authentication 20468 .IP \[bu] 2 20469 \f[C]--rc-user gui\f[R] 20470 .IP \[bu] 2 20471 \f[C]--rc-pass <random password>\f[R] 20472 .IP \[bu] 2 20473 \f[C]--rc-serve\f[R] 20474 .PP 20475 These flags can be overridden as desired. 20476 .PP 20477 See also the rclone rcd 20478 documentation (https://rclone.org/commands/rclone_rcd/). 20479 .SS Example: Running a public GUI 20480 .PP 20481 For example the GUI could be served on a public port over SSL using an 20482 htpasswd file using the following flags: 20483 .IP \[bu] 2 20484 \f[C]--rc-web-gui\f[R] 20485 .IP \[bu] 2 20486 \f[C]--rc-addr :443\f[R] 20487 .IP \[bu] 2 20488 \f[C]--rc-htpasswd /path/to/htpasswd\f[R] 20489 .IP \[bu] 2 20490 \f[C]--rc-cert /path/to/ssl.crt\f[R] 20491 .IP \[bu] 2 20492 \f[C]--rc-key /path/to/ssl.key\f[R] 20493 .SS Example: Running a GUI behind a proxy 20494 .PP 20495 If you want to run the GUI behind a proxy at \f[C]/rclone\f[R] you could 20496 use these flags: 20497 .IP \[bu] 2 20498 \f[C]--rc-web-gui\f[R] 20499 .IP \[bu] 2 20500 \f[C]--rc-baseurl rclone\f[R] 20501 .IP \[bu] 2 20502 \f[C]--rc-htpasswd /path/to/htpasswd\f[R] 20503 .PP 20504 Or instead of htpasswd if you just want a single user and password: 20505 .IP \[bu] 2 20506 \f[C]--rc-user me\f[R] 20507 .IP \[bu] 2 20508 \f[C]--rc-pass mypassword\f[R] 20509 .SS Project 20510 .PP 20511 The GUI is being developed in the: artpar/artpar-webui-react 20512 repository (https://github.com/artpar/artpar-webui-react). 20513 .PP 20514 Bug reports and contributions are very welcome :-) 20515 .PP 20516 If you have questions then please ask them on the rclone 20517 forum (https://forum.rclone.org/). 20518 .SH Remote controlling rclone with its API 20519 .PP 20520 If rclone is run with the \f[C]--rc\f[R] flag then it starts an HTTP 20521 server which can be used to remote control rclone using its API. 20522 .PP 20523 You can either use the rc command to access the API or use HTTP 20524 directly. 20525 .PP 20526 If you just want to run a remote control then see the 20527 rcd (https://rclone.org/commands/rclone_rcd/) command. 20528 .SS Supported parameters 20529 .SS --rc 20530 .PP 20531 Flag to start the http server listen on remote requests 20532 .SS --rc-addr=IP 20533 .PP 20534 IPaddress:Port or :Port to bind server to. 20535 (default \[dq]localhost:5572\[dq]) 20536 .SS --rc-cert=KEY 20537 .PP 20538 SSL PEM key (concatenation of certificate and CA certificate) 20539 .SS --rc-client-ca=PATH 20540 .PP 20541 Client certificate authority to verify clients with 20542 .SS --rc-htpasswd=PATH 20543 .PP 20544 htpasswd file - if not provided no authentication is done 20545 .SS --rc-key=PATH 20546 .PP 20547 SSL PEM Private key 20548 .SS --rc-max-header-bytes=VALUE 20549 .PP 20550 Maximum size of request header (default 4096) 20551 .SS --rc-min-tls-version=VALUE 20552 .PP 20553 The minimum TLS version that is acceptable. 20554 Valid values are \[dq]tls1.0\[dq], \[dq]tls1.1\[dq], \[dq]tls1.2\[dq] 20555 and \[dq]tls1.3\[dq] (default \[dq]tls1.0\[dq]). 20556 .SS --rc-user=VALUE 20557 .PP 20558 User name for authentication. 20559 .SS --rc-pass=VALUE 20560 .PP 20561 Password for authentication. 20562 .SS --rc-realm=VALUE 20563 .PP 20564 Realm for authentication (default \[dq]rclone\[dq]) 20565 .SS --rc-server-read-timeout=DURATION 20566 .PP 20567 Timeout for server reading data (default 1h0m0s) 20568 .SS --rc-server-write-timeout=DURATION 20569 .PP 20570 Timeout for server writing data (default 1h0m0s) 20571 .SS --rc-serve 20572 .PP 20573 Enable the serving of remote objects via the HTTP interface. 20574 This means objects will be accessible at http://127.0.0.1:5572/ by 20575 default, so you can browse to http://127.0.0.1:5572/ or 20576 http://127.0.0.1:5572/* to see a listing of the remotes. 20577 Objects may be requested from remotes using this syntax 20578 http://127.0.0.1:5572/[remote:path]/path/to/object 20579 .PP 20580 Default Off. 20581 .SS --rc-files /path/to/directory 20582 .PP 20583 Path to local files to serve on the HTTP server. 20584 .PP 20585 If this is set then rclone will serve the files in that directory. 20586 It will also open the root in the web browser if specified. 20587 This is for implementing browser based GUIs for rclone functions. 20588 .PP 20589 If \f[C]--rc-user\f[R] or \f[C]--rc-pass\f[R] is set then the URL that 20590 is opened will have the authorization in the URL in the 20591 \f[C]http://user:pass\[at]localhost/\f[R] style. 20592 .PP 20593 Default Off. 20594 .SS --rc-enable-metrics 20595 .PP 20596 Enable OpenMetrics/Prometheus compatible endpoint at \f[C]/metrics\f[R]. 20597 .PP 20598 Default Off. 20599 .SS --rc-web-gui 20600 .PP 20601 Set this flag to serve the default web gui on the same port as rclone. 20602 .PP 20603 Default Off. 20604 .SS --rc-allow-origin 20605 .PP 20606 Set the allowed Access-Control-Allow-Origin for rc requests. 20607 .PP 20608 Can be used with --rc-web-gui if the rclone is running on different IP 20609 than the web-gui. 20610 .PP 20611 Default is IP address on which rc is running. 20612 .SS --rc-web-fetch-url 20613 .PP 20614 Set the URL to fetch the rclone-web-gui files from. 20615 .PP 20616 Default 20617 https://api.github.com/repos/artpar/artpar-webui-react/releases/latest. 20618 .SS --rc-web-gui-update 20619 .PP 20620 Set this flag to check and update rclone-webui-react from the 20621 rc-web-fetch-url. 20622 .PP 20623 Default Off. 20624 .SS --rc-web-gui-force-update 20625 .PP 20626 Set this flag to force update rclone-webui-react from the 20627 rc-web-fetch-url. 20628 .PP 20629 Default Off. 20630 .SS --rc-web-gui-no-open-browser 20631 .PP 20632 Set this flag to disable opening browser automatically when using 20633 web-gui. 20634 .PP 20635 Default Off. 20636 .SS --rc-job-expire-duration=DURATION 20637 .PP 20638 Expire finished async jobs older than DURATION (default 60s). 20639 .SS --rc-job-expire-interval=DURATION 20640 .PP 20641 Interval duration to check for expired async jobs (default 10s). 20642 .SS --rc-no-auth 20643 .PP 20644 By default rclone will require authorisation to have been set up on the 20645 rc interface in order to use any methods which access any rclone 20646 remotes. 20647 Eg \f[C]operations/list\f[R] is denied as it involved creating a remote 20648 as is \f[C]sync/copy\f[R]. 20649 .PP 20650 If this is set then no authorisation will be required on the server to 20651 use these methods. 20652 The alternative is to use \f[C]--rc-user\f[R] and \f[C]--rc-pass\f[R] 20653 and use these credentials in the request. 20654 .PP 20655 Default Off. 20656 .SS --rc-baseurl 20657 .PP 20658 Prefix for URLs. 20659 .PP 20660 Default is root 20661 .SS --rc-template 20662 .PP 20663 User-specified template. 20664 .SS Accessing the remote control via the rclone rc command 20665 .PP 20666 Rclone itself implements the remote control protocol in its 20667 \f[C]rclone rc\f[R] command. 20668 .PP 20669 You can use it like this 20670 .IP 20671 .nf 20672 \f[C] 20673 $ rclone rc rc/noop param1=one param2=two 20674 { 20675 \[dq]param1\[dq]: \[dq]one\[dq], 20676 \[dq]param2\[dq]: \[dq]two\[dq] 20677 } 20678 \f[R] 20679 .fi 20680 .PP 20681 Run \f[C]rclone rc\f[R] on its own to see the help for the installed 20682 remote control commands. 20683 .SS JSON input 20684 .PP 20685 \f[C]rclone rc\f[R] also supports a \f[C]--json\f[R] flag which can be 20686 used to send more complicated input parameters. 20687 .IP 20688 .nf 20689 \f[C] 20690 $ rclone rc --json \[aq]{ \[dq]p1\[dq]: [1,\[dq]2\[dq],null,4], \[dq]p2\[dq]: { \[dq]a\[dq]:1, \[dq]b\[dq]:2 } }\[aq] rc/noop 20691 { 20692 \[dq]p1\[dq]: [ 20693 1, 20694 \[dq]2\[dq], 20695 null, 20696 4 20697 ], 20698 \[dq]p2\[dq]: { 20699 \[dq]a\[dq]: 1, 20700 \[dq]b\[dq]: 2 20701 } 20702 } 20703 \f[R] 20704 .fi 20705 .PP 20706 If the parameter being passed is an object then it can be passed as a 20707 JSON string rather than using the \f[C]--json\f[R] flag which simplifies 20708 the command line. 20709 .IP 20710 .nf 20711 \f[C] 20712 rclone rc operations/list fs=/tmp remote=test opt=\[aq]{\[dq]showHash\[dq]: true}\[aq] 20713 \f[R] 20714 .fi 20715 .PP 20716 Rather than 20717 .IP 20718 .nf 20719 \f[C] 20720 rclone rc operations/list --json \[aq]{\[dq]fs\[dq]: \[dq]/tmp\[dq], \[dq]remote\[dq]: \[dq]test\[dq], \[dq]opt\[dq]: {\[dq]showHash\[dq]: true}}\[aq] 20721 \f[R] 20722 .fi 20723 .SS Special parameters 20724 .PP 20725 The rc interface supports some special parameters which apply to 20726 \f[B]all\f[R] commands. 20727 These start with \f[C]_\f[R] to show they are different. 20728 .SS Running asynchronous jobs with _async = true 20729 .PP 20730 Each rc call is classified as a job and it is assigned its own id. 20731 By default jobs are executed immediately as they are created or 20732 synchronously. 20733 .PP 20734 If \f[C]_async\f[R] has a true value when supplied to an rc call then it 20735 will return immediately with a job id and the task will be run in the 20736 background. 20737 The \f[C]job/status\f[R] call can be used to get information of the 20738 background job. 20739 The job can be queried for up to 1 minute after it has finished. 20740 .PP 20741 It is recommended that potentially long running jobs, e.g. 20742 \f[C]sync/sync\f[R], \f[C]sync/copy\f[R], \f[C]sync/move\f[R], 20743 \f[C]operations/purge\f[R] are run with the \f[C]_async\f[R] flag to 20744 avoid any potential problems with the HTTP request and response timing 20745 out. 20746 .PP 20747 Starting a job with the \f[C]_async\f[R] flag: 20748 .IP 20749 .nf 20750 \f[C] 20751 $ rclone rc --json \[aq]{ \[dq]p1\[dq]: [1,\[dq]2\[dq],null,4], \[dq]p2\[dq]: { \[dq]a\[dq]:1, \[dq]b\[dq]:2 }, \[dq]_async\[dq]: true }\[aq] rc/noop 20752 { 20753 \[dq]jobid\[dq]: 2 20754 } 20755 \f[R] 20756 .fi 20757 .PP 20758 Query the status to see if the job has finished. 20759 For more information on the meaning of these return parameters see the 20760 \f[C]job/status\f[R] call. 20761 .IP 20762 .nf 20763 \f[C] 20764 $ rclone rc --json \[aq]{ \[dq]jobid\[dq]:2 }\[aq] job/status 20765 { 20766 \[dq]duration\[dq]: 0.000124163, 20767 \[dq]endTime\[dq]: \[dq]2018-10-27T11:38:07.911245881+01:00\[dq], 20768 \[dq]error\[dq]: \[dq]\[dq], 20769 \[dq]finished\[dq]: true, 20770 \[dq]id\[dq]: 2, 20771 \[dq]output\[dq]: { 20772 \[dq]_async\[dq]: true, 20773 \[dq]p1\[dq]: [ 20774 1, 20775 \[dq]2\[dq], 20776 null, 20777 4 20778 ], 20779 \[dq]p2\[dq]: { 20780 \[dq]a\[dq]: 1, 20781 \[dq]b\[dq]: 2 20782 } 20783 }, 20784 \[dq]startTime\[dq]: \[dq]2018-10-27T11:38:07.911121728+01:00\[dq], 20785 \[dq]success\[dq]: true 20786 } 20787 \f[R] 20788 .fi 20789 .PP 20790 \f[C]job/list\f[R] can be used to show the running or recently completed 20791 jobs 20792 .IP 20793 .nf 20794 \f[C] 20795 $ rclone rc job/list 20796 { 20797 \[dq]jobids\[dq]: [ 20798 2 20799 ] 20800 } 20801 \f[R] 20802 .fi 20803 .SS Setting config flags with _config 20804 .PP 20805 If you wish to set config (the equivalent of the global flags) for the 20806 duration of an rc call only then pass in the \f[C]_config\f[R] 20807 parameter. 20808 .PP 20809 This should be in the same format as the \f[C]config\f[R] key returned 20810 by options/get. 20811 .PP 20812 For example, if you wished to run a sync with the \f[C]--checksum\f[R] 20813 parameter, you would pass this parameter in your JSON blob. 20814 .IP 20815 .nf 20816 \f[C] 20817 \[dq]_config\[dq]:{\[dq]CheckSum\[dq]: true} 20818 \f[R] 20819 .fi 20820 .PP 20821 If using \f[C]rclone rc\f[R] this could be passed as 20822 .IP 20823 .nf 20824 \f[C] 20825 rclone rc sync/sync ... _config=\[aq]{\[dq]CheckSum\[dq]: true}\[aq] 20826 \f[R] 20827 .fi 20828 .PP 20829 Any config parameters you don\[aq]t set will inherit the global defaults 20830 which were set with command line flags or environment variables. 20831 .PP 20832 Note that it is possible to set some values as strings or integers - see 20833 data types for more info. 20834 Here is an example setting the equivalent of \f[C]--buffer-size\f[R] in 20835 string or integer format. 20836 .IP 20837 .nf 20838 \f[C] 20839 \[dq]_config\[dq]:{\[dq]BufferSize\[dq]: \[dq]42M\[dq]} 20840 \[dq]_config\[dq]:{\[dq]BufferSize\[dq]: 44040192} 20841 \f[R] 20842 .fi 20843 .PP 20844 If you wish to check the \f[C]_config\f[R] assignment has worked 20845 properly then calling \f[C]options/local\f[R] will show what the value 20846 got set to. 20847 .SS Setting filter flags with _filter 20848 .PP 20849 If you wish to set filters for the duration of an rc call only then pass 20850 in the \f[C]_filter\f[R] parameter. 20851 .PP 20852 This should be in the same format as the \f[C]filter\f[R] key returned 20853 by options/get. 20854 .PP 20855 For example, if you wished to run a sync with these flags 20856 .IP 20857 .nf 20858 \f[C] 20859 --max-size 1M --max-age 42s --include \[dq]a\[dq] --include \[dq]b\[dq] 20860 \f[R] 20861 .fi 20862 .PP 20863 you would pass this parameter in your JSON blob. 20864 .IP 20865 .nf 20866 \f[C] 20867 \[dq]_filter\[dq]:{\[dq]MaxSize\[dq]:\[dq]1M\[dq], \[dq]IncludeRule\[dq]:[\[dq]a\[dq],\[dq]b\[dq]], \[dq]MaxAge\[dq]:\[dq]42s\[dq]} 20868 \f[R] 20869 .fi 20870 .PP 20871 If using \f[C]rclone rc\f[R] this could be passed as 20872 .IP 20873 .nf 20874 \f[C] 20875 rclone rc ... _filter=\[aq]{\[dq]MaxSize\[dq]:\[dq]1M\[dq], \[dq]IncludeRule\[dq]:[\[dq]a\[dq],\[dq]b\[dq]], \[dq]MaxAge\[dq]:\[dq]42s\[dq]}\[aq] 20876 \f[R] 20877 .fi 20878 .PP 20879 Any filter parameters you don\[aq]t set will inherit the global defaults 20880 which were set with command line flags or environment variables. 20881 .PP 20882 Note that it is possible to set some values as strings or integers - see 20883 data types for more info. 20884 Here is an example setting the equivalent of \f[C]--buffer-size\f[R] in 20885 string or integer format. 20886 .IP 20887 .nf 20888 \f[C] 20889 \[dq]_filter\[dq]:{\[dq]MinSize\[dq]: \[dq]42M\[dq]} 20890 \[dq]_filter\[dq]:{\[dq]MinSize\[dq]: 44040192} 20891 \f[R] 20892 .fi 20893 .PP 20894 If you wish to check the \f[C]_filter\f[R] assignment has worked 20895 properly then calling \f[C]options/local\f[R] will show what the value 20896 got set to. 20897 .SS Assigning operations to groups with _group = value 20898 .PP 20899 Each rc call has its own stats group for tracking its metrics. 20900 By default grouping is done by the composite group name from prefix 20901 \f[C]job/\f[R] and id of the job like so \f[C]job/1\f[R]. 20902 .PP 20903 If \f[C]_group\f[R] has a value then stats for that request will be 20904 grouped under that value. 20905 This allows caller to group stats under their own name. 20906 .PP 20907 Stats for specific group can be accessed by passing \f[C]group\f[R] to 20908 \f[C]core/stats\f[R]: 20909 .IP 20910 .nf 20911 \f[C] 20912 $ rclone rc --json \[aq]{ \[dq]group\[dq]: \[dq]job/1\[dq] }\[aq] core/stats 20913 { 20914 \[dq]speed\[dq]: 12345 20915 ... 20916 } 20917 \f[R] 20918 .fi 20919 .SS Data types 20920 .PP 20921 When the API returns types, these will mostly be straight forward 20922 integer, string or boolean types. 20923 .PP 20924 However some of the types returned by the options/get call and taken by 20925 the options/set calls as well as the \f[C]vfsOpt\f[R], 20926 \f[C]mountOpt\f[R] and the \f[C]_config\f[R] parameters. 20927 .IP \[bu] 2 20928 \f[C]Duration\f[R] - these are returned as an integer duration in 20929 nanoseconds. 20930 They may be set as an integer, or they may be set with time string, eg 20931 \[dq]5s\[dq]. 20932 See the options section (https://rclone.org/docs/#options) for more 20933 info. 20934 .IP \[bu] 2 20935 \f[C]Size\f[R] - these are returned as an integer number of bytes. 20936 They may be set as an integer or they may be set with a size suffix 20937 string, eg \[dq]10M\[dq]. 20938 See the options section (https://rclone.org/docs/#options) for more 20939 info. 20940 .IP \[bu] 2 20941 Enumerated type (such as \f[C]CutoffMode\f[R], \f[C]DumpFlags\f[R], 20942 \f[C]LogLevel\f[R], \f[C]VfsCacheMode\f[R] - these will be returned as 20943 an integer and may be set as an integer but more conveniently they can 20944 be set as a string, eg \[dq]HARD\[dq] for \f[C]CutoffMode\f[R] or 20945 \f[C]DEBUG\f[R] for \f[C]LogLevel\f[R]. 20946 .IP \[bu] 2 20947 \f[C]BandwidthSpec\f[R] - this will be set and returned as a string, eg 20948 \[dq]1M\[dq]. 20949 .SS Specifying remotes to work on 20950 .PP 20951 Remotes are specified with the \f[C]fs=\f[R], \f[C]srcFs=\f[R], 20952 \f[C]dstFs=\f[R] parameters depending on the command being used. 20953 .PP 20954 The parameters can be a string as per the rest of rclone, eg 20955 \f[C]s3:bucket/path\f[R] or \f[C]:sftp:/my/dir\f[R]. 20956 They can also be specified as JSON blobs. 20957 .PP 20958 If specifying a JSON blob it should be a object mapping strings to 20959 strings. 20960 These values will be used to configure the remote. 20961 There are 3 special values which may be set: 20962 .IP \[bu] 2 20963 \f[C]type\f[R] - set to \f[C]type\f[R] to specify a remote called 20964 \f[C]:type:\f[R] 20965 .IP \[bu] 2 20966 \f[C]_name\f[R] - set to \f[C]name\f[R] to specify a remote called 20967 \f[C]name:\f[R] 20968 .IP \[bu] 2 20969 \f[C]_root\f[R] - sets the root of the remote - may be empty 20970 .PP 20971 One of \f[C]_name\f[R] or \f[C]type\f[R] should normally be set. 20972 If the \f[C]local\f[R] backend is desired then \f[C]type\f[R] should be 20973 set to \f[C]local\f[R]. 20974 If \f[C]_root\f[R] isn\[aq]t specified then it defaults to the root of 20975 the remote. 20976 .PP 20977 For example this JSON is equivalent to \f[C]remote:/tmp\f[R] 20978 .IP 20979 .nf 20980 \f[C] 20981 { 20982 \[dq]_name\[dq]: \[dq]remote\[dq], 20983 \[dq]_path\[dq]: \[dq]/tmp\[dq] 20984 } 20985 \f[R] 20986 .fi 20987 .PP 20988 And this is equivalent to 20989 \f[C]:sftp,host=\[aq]example.com\[aq]:/tmp\f[R] 20990 .IP 20991 .nf 20992 \f[C] 20993 { 20994 \[dq]type\[dq]: \[dq]sftp\[dq], 20995 \[dq]host\[dq]: \[dq]example.com\[dq], 20996 \[dq]_path\[dq]: \[dq]/tmp\[dq] 20997 } 20998 \f[R] 20999 .fi 21000 .PP 21001 And this is equivalent to \f[C]/tmp/dir\f[R] 21002 .IP 21003 .nf 21004 \f[C] 21005 { 21006 type = \[dq]local\[dq], 21007 _ path = \[dq]/tmp/dir\[dq] 21008 } 21009 \f[R] 21010 .fi 21011 .SS Supported commands 21012 .SS backend/command: Runs a backend command. 21013 .PP 21014 This takes the following parameters: 21015 .IP \[bu] 2 21016 command - a string with the command name 21017 .IP \[bu] 2 21018 fs - a remote name string e.g. 21019 \[dq]drive:\[dq] 21020 .IP \[bu] 2 21021 arg - a list of arguments for the backend command 21022 .IP \[bu] 2 21023 opt - a map of string to string of options 21024 .PP 21025 Returns: 21026 .IP \[bu] 2 21027 result - result from the backend command 21028 .PP 21029 Example: 21030 .IP 21031 .nf 21032 \f[C] 21033 rclone rc backend/command command=noop fs=. -o echo=yes -o blue -a path1 -a path2 21034 \f[R] 21035 .fi 21036 .PP 21037 Returns 21038 .IP 21039 .nf 21040 \f[C] 21041 { 21042 \[dq]result\[dq]: { 21043 \[dq]arg\[dq]: [ 21044 \[dq]path1\[dq], 21045 \[dq]path2\[dq] 21046 ], 21047 \[dq]name\[dq]: \[dq]noop\[dq], 21048 \[dq]opt\[dq]: { 21049 \[dq]blue\[dq]: \[dq]\[dq], 21050 \[dq]echo\[dq]: \[dq]yes\[dq] 21051 } 21052 } 21053 } 21054 \f[R] 21055 .fi 21056 .PP 21057 Note that this is the direct equivalent of using this \[dq]backend\[dq] 21058 command: 21059 .IP 21060 .nf 21061 \f[C] 21062 rclone backend noop . -o echo=yes -o blue path1 path2 21063 \f[R] 21064 .fi 21065 .PP 21066 Note that arguments must be preceded by the \[dq]-a\[dq] flag 21067 .PP 21068 See the backend (https://rclone.org/commands/rclone_backend/) command 21069 for more information. 21070 .PP 21071 \f[B]Authentication is required for this call.\f[R] 21072 .SS cache/expire: Purge a remote from cache 21073 .PP 21074 Purge a remote from the cache backend. 21075 Supports either a directory or a file. 21076 Params: - remote = path to remote (required) - withData = true/false to 21077 delete cached data (chunks) as well (optional) 21078 .PP 21079 Eg 21080 .IP 21081 .nf 21082 \f[C] 21083 rclone rc cache/expire remote=path/to/sub/folder/ 21084 rclone rc cache/expire remote=/ withData=true 21085 \f[R] 21086 .fi 21087 .SS cache/fetch: Fetch file chunks 21088 .PP 21089 Ensure the specified file chunks are cached on disk. 21090 .PP 21091 The chunks= parameter specifies the file chunks to check. 21092 It takes a comma separated list of array slice indices. 21093 The slice indices are similar to Python slices: start[:end] 21094 .PP 21095 start is the 0 based chunk number from the beginning of the file to 21096 fetch inclusive. 21097 end is 0 based chunk number from the beginning of the file to fetch 21098 exclusive. 21099 Both values can be negative, in which case they count from the back of 21100 the file. 21101 The value \[dq]-5:\[dq] represents the last 5 chunks of a file. 21102 .PP 21103 Some valid examples are: \[dq]:5,-5:\[dq] -> the first and last five 21104 chunks \[dq]0,-2\[dq] -> the first and the second last chunk 21105 \[dq]0:10\[dq] -> the first ten chunks 21106 .PP 21107 Any parameter with a key that starts with \[dq]file\[dq] can be used to 21108 specify files to fetch, e.g. 21109 .IP 21110 .nf 21111 \f[C] 21112 rclone rc cache/fetch chunks=0 file=hello file2=home/goodbye 21113 \f[R] 21114 .fi 21115 .PP 21116 File names will automatically be encrypted when the a crypt remote is 21117 used on top of the cache. 21118 .SS cache/stats: Get cache stats 21119 .PP 21120 Show statistics for the cache remote. 21121 .SS config/create: create the config for a remote. 21122 .PP 21123 This takes the following parameters: 21124 .IP \[bu] 2 21125 name - name of remote 21126 .IP \[bu] 2 21127 parameters - a map of { \[dq]key\[dq]: \[dq]value\[dq] } pairs 21128 .IP \[bu] 2 21129 type - type of the new remote 21130 .IP \[bu] 2 21131 opt - a dictionary of options to control the configuration 21132 .RS 2 21133 .IP \[bu] 2 21134 obscure - declare passwords are plain and need obscuring 21135 .IP \[bu] 2 21136 noObscure - declare passwords are already obscured and don\[aq]t need 21137 obscuring 21138 .IP \[bu] 2 21139 nonInteractive - don\[aq]t interact with a user, return questions 21140 .IP \[bu] 2 21141 continue - continue the config process with an answer 21142 .IP \[bu] 2 21143 all - ask all the config questions not just the post config ones 21144 .IP \[bu] 2 21145 state - state to restart with - used with continue 21146 .IP \[bu] 2 21147 result - result to restart with - used with continue 21148 .RE 21149 .PP 21150 See the config 21151 create (https://rclone.org/commands/rclone_config_create/) command for 21152 more information on the above. 21153 .PP 21154 \f[B]Authentication is required for this call.\f[R] 21155 .SS config/delete: Delete a remote in the config file. 21156 .PP 21157 Parameters: 21158 .IP \[bu] 2 21159 name - name of remote to delete 21160 .PP 21161 See the config 21162 delete (https://rclone.org/commands/rclone_config_delete/) command for 21163 more information on the above. 21164 .PP 21165 \f[B]Authentication is required for this call.\f[R] 21166 .SS config/dump: Dumps the config file. 21167 .PP 21168 Returns a JSON object: - key: value 21169 .PP 21170 Where keys are remote names and values are the config parameters. 21171 .PP 21172 See the config dump (https://rclone.org/commands/rclone_config_dump/) 21173 command for more information on the above. 21174 .PP 21175 \f[B]Authentication is required for this call.\f[R] 21176 .SS config/get: Get a remote in the config file. 21177 .PP 21178 Parameters: 21179 .IP \[bu] 2 21180 name - name of remote to get 21181 .PP 21182 See the config dump (https://rclone.org/commands/rclone_config_dump/) 21183 command for more information on the above. 21184 .PP 21185 \f[B]Authentication is required for this call.\f[R] 21186 .SS config/listremotes: Lists the remotes in the config file and defined in environment variables. 21187 .PP 21188 Returns - remotes - array of remote names 21189 .PP 21190 See the listremotes (https://rclone.org/commands/rclone_listremotes/) 21191 command for more information on the above. 21192 .PP 21193 \f[B]Authentication is required for this call.\f[R] 21194 .SS config/password: password the config for a remote. 21195 .PP 21196 This takes the following parameters: 21197 .IP \[bu] 2 21198 name - name of remote 21199 .IP \[bu] 2 21200 parameters - a map of { \[dq]key\[dq]: \[dq]value\[dq] } pairs 21201 .PP 21202 See the config 21203 password (https://rclone.org/commands/rclone_config_password/) command 21204 for more information on the above. 21205 .PP 21206 \f[B]Authentication is required for this call.\f[R] 21207 .SS config/paths: Reads the config file path and other important paths. 21208 .PP 21209 Returns a JSON object with the following keys: 21210 .IP \[bu] 2 21211 config: path to config file 21212 .IP \[bu] 2 21213 cache: path to root of cache directory 21214 .IP \[bu] 2 21215 temp: path to root of temporary directory 21216 .PP 21217 Eg 21218 .IP 21219 .nf 21220 \f[C] 21221 { 21222 \[dq]cache\[dq]: \[dq]/home/USER/.cache/rclone\[dq], 21223 \[dq]config\[dq]: \[dq]/home/USER/.rclone.conf\[dq], 21224 \[dq]temp\[dq]: \[dq]/tmp\[dq] 21225 } 21226 \f[R] 21227 .fi 21228 .PP 21229 See the config paths (https://rclone.org/commands/rclone_config_paths/) 21230 command for more information on the above. 21231 .PP 21232 \f[B]Authentication is required for this call.\f[R] 21233 .SS config/providers: Shows how providers are configured in the config file. 21234 .PP 21235 Returns a JSON object: - providers - array of objects 21236 .PP 21237 See the config 21238 providers (https://rclone.org/commands/rclone_config_providers/) command 21239 for more information on the above. 21240 .PP 21241 \f[B]Authentication is required for this call.\f[R] 21242 .SS config/setpath: Set the path of the config file 21243 .PP 21244 Parameters: 21245 .IP \[bu] 2 21246 path - path to the config file to use 21247 .PP 21248 \f[B]Authentication is required for this call.\f[R] 21249 .SS config/update: update the config for a remote. 21250 .PP 21251 This takes the following parameters: 21252 .IP \[bu] 2 21253 name - name of remote 21254 .IP \[bu] 2 21255 parameters - a map of { \[dq]key\[dq]: \[dq]value\[dq] } pairs 21256 .IP \[bu] 2 21257 opt - a dictionary of options to control the configuration 21258 .RS 2 21259 .IP \[bu] 2 21260 obscure - declare passwords are plain and need obscuring 21261 .IP \[bu] 2 21262 noObscure - declare passwords are already obscured and don\[aq]t need 21263 obscuring 21264 .IP \[bu] 2 21265 nonInteractive - don\[aq]t interact with a user, return questions 21266 .IP \[bu] 2 21267 continue - continue the config process with an answer 21268 .IP \[bu] 2 21269 all - ask all the config questions not just the post config ones 21270 .IP \[bu] 2 21271 state - state to restart with - used with continue 21272 .IP \[bu] 2 21273 result - result to restart with - used with continue 21274 .RE 21275 .PP 21276 See the config 21277 update (https://rclone.org/commands/rclone_config_update/) command for 21278 more information on the above. 21279 .PP 21280 \f[B]Authentication is required for this call.\f[R] 21281 .SS core/bwlimit: Set the bandwidth limit. 21282 .PP 21283 This sets the bandwidth limit to the string passed in. 21284 This should be a single bandwidth limit entry or a pair of 21285 upload:download bandwidth. 21286 .PP 21287 Eg 21288 .IP 21289 .nf 21290 \f[C] 21291 rclone rc core/bwlimit rate=off 21292 { 21293 \[dq]bytesPerSecond\[dq]: -1, 21294 \[dq]bytesPerSecondTx\[dq]: -1, 21295 \[dq]bytesPerSecondRx\[dq]: -1, 21296 \[dq]rate\[dq]: \[dq]off\[dq] 21297 } 21298 rclone rc core/bwlimit rate=1M 21299 { 21300 \[dq]bytesPerSecond\[dq]: 1048576, 21301 \[dq]bytesPerSecondTx\[dq]: 1048576, 21302 \[dq]bytesPerSecondRx\[dq]: 1048576, 21303 \[dq]rate\[dq]: \[dq]1M\[dq] 21304 } 21305 rclone rc core/bwlimit rate=1M:100k 21306 { 21307 \[dq]bytesPerSecond\[dq]: 1048576, 21308 \[dq]bytesPerSecondTx\[dq]: 1048576, 21309 \[dq]bytesPerSecondRx\[dq]: 131072, 21310 \[dq]rate\[dq]: \[dq]1M\[dq] 21311 } 21312 \f[R] 21313 .fi 21314 .PP 21315 If the rate parameter is not supplied then the bandwidth is queried 21316 .IP 21317 .nf 21318 \f[C] 21319 rclone rc core/bwlimit 21320 { 21321 \[dq]bytesPerSecond\[dq]: 1048576, 21322 \[dq]bytesPerSecondTx\[dq]: 1048576, 21323 \[dq]bytesPerSecondRx\[dq]: 1048576, 21324 \[dq]rate\[dq]: \[dq]1M\[dq] 21325 } 21326 \f[R] 21327 .fi 21328 .PP 21329 The format of the parameter is exactly the same as passed to --bwlimit 21330 except only one bandwidth may be specified. 21331 .PP 21332 In either case \[dq]rate\[dq] is returned as a human-readable string, 21333 and \[dq]bytesPerSecond\[dq] is returned as a number. 21334 .SS core/command: Run a rclone terminal command over rc. 21335 .PP 21336 This takes the following parameters: 21337 .IP \[bu] 2 21338 command - a string with the command name. 21339 .IP \[bu] 2 21340 arg - a list of arguments for the backend command. 21341 .IP \[bu] 2 21342 opt - a map of string to string of options. 21343 .IP \[bu] 2 21344 returnType - one of (\[dq]COMBINED_OUTPUT\[dq], \[dq]STREAM\[dq], 21345 \[dq]STREAM_ONLY_STDOUT\[dq], \[dq]STREAM_ONLY_STDERR\[dq]). 21346 .RS 2 21347 .IP \[bu] 2 21348 Defaults to \[dq]COMBINED_OUTPUT\[dq] if not set. 21349 .IP \[bu] 2 21350 The STREAM returnTypes will write the output to the body of the HTTP 21351 message. 21352 .IP \[bu] 2 21353 The COMBINED_OUTPUT will write the output to the \[dq]result\[dq] 21354 parameter. 21355 .RE 21356 .PP 21357 Returns: 21358 .IP \[bu] 2 21359 result - result from the backend command. 21360 .RS 2 21361 .IP \[bu] 2 21362 Only set when using returnType \[dq]COMBINED_OUTPUT\[dq]. 21363 .RE 21364 .IP \[bu] 2 21365 error - set if rclone exits with an error code. 21366 .IP \[bu] 2 21367 returnType - one of (\[dq]COMBINED_OUTPUT\[dq], \[dq]STREAM\[dq], 21368 \[dq]STREAM_ONLY_STDOUT\[dq], \[dq]STREAM_ONLY_STDERR\[dq]). 21369 .PP 21370 Example: 21371 .IP 21372 .nf 21373 \f[C] 21374 rclone rc core/command command=ls -a mydrive:/ -o max-depth=1 21375 rclone rc core/command -a ls -a mydrive:/ -o max-depth=1 21376 \f[R] 21377 .fi 21378 .PP 21379 Returns: 21380 .IP 21381 .nf 21382 \f[C] 21383 { 21384 \[dq]error\[dq]: false, 21385 \[dq]result\[dq]: \[dq]<Raw command line output>\[dq] 21386 } 21387 21388 OR 21389 { 21390 \[dq]error\[dq]: true, 21391 \[dq]result\[dq]: \[dq]<Raw command line output>\[dq] 21392 } 21393 \f[R] 21394 .fi 21395 .PP 21396 \f[B]Authentication is required for this call.\f[R] 21397 .SS core/du: Returns disk usage of a locally attached disk. 21398 .PP 21399 This returns the disk usage for the local directory passed in as dir. 21400 .PP 21401 If the directory is not passed in, it defaults to the directory pointed 21402 to by --cache-dir. 21403 .IP \[bu] 2 21404 dir - string (optional) 21405 .PP 21406 Returns: 21407 .IP 21408 .nf 21409 \f[C] 21410 { 21411 \[dq]dir\[dq]: \[dq]/\[dq], 21412 \[dq]info\[dq]: { 21413 \[dq]Available\[dq]: 361769115648, 21414 \[dq]Free\[dq]: 361785892864, 21415 \[dq]Total\[dq]: 982141468672 21416 } 21417 } 21418 \f[R] 21419 .fi 21420 .SS core/gc: Runs a garbage collection. 21421 .PP 21422 This tells the go runtime to do a garbage collection run. 21423 It isn\[aq]t necessary to call this normally, but it can be useful for 21424 debugging memory problems. 21425 .SS core/group-list: Returns list of stats. 21426 .PP 21427 This returns list of stats groups currently in memory. 21428 .PP 21429 Returns the following values: 21430 .IP 21431 .nf 21432 \f[C] 21433 { 21434 \[dq]groups\[dq]: an array of group names: 21435 [ 21436 \[dq]group1\[dq], 21437 \[dq]group2\[dq], 21438 ... 21439 ] 21440 } 21441 \f[R] 21442 .fi 21443 .SS core/memstats: Returns the memory statistics 21444 .PP 21445 This returns the memory statistics of the running program. 21446 What the values mean are explained in the go docs: 21447 https://golang.org/pkg/runtime/#MemStats 21448 .PP 21449 The most interesting values for most people are: 21450 .IP \[bu] 2 21451 HeapAlloc - this is the amount of memory rclone is actually using 21452 .IP \[bu] 2 21453 HeapSys - this is the amount of memory rclone has obtained from the OS 21454 .IP \[bu] 2 21455 Sys - this is the total amount of memory requested from the OS 21456 .RS 2 21457 .IP \[bu] 2 21458 It is virtual memory so may include unused memory 21459 .RE 21460 .SS core/obscure: Obscures a string passed in. 21461 .PP 21462 Pass a clear string and rclone will obscure it for the config file: - 21463 clear - string 21464 .PP 21465 Returns: - obscured - string 21466 .SS core/pid: Return PID of current process 21467 .PP 21468 This returns PID of current process. 21469 Useful for stopping rclone process. 21470 .SS core/quit: Terminates the app. 21471 .PP 21472 (Optional) Pass an exit code to be used for terminating the app: - 21473 exitCode - int 21474 .SS core/stats: Returns stats about current transfers. 21475 .PP 21476 This returns all available stats: 21477 .IP 21478 .nf 21479 \f[C] 21480 rclone rc core/stats 21481 \f[R] 21482 .fi 21483 .PP 21484 If group is not provided then summed up stats for all groups will be 21485 returned. 21486 .PP 21487 Parameters 21488 .IP \[bu] 2 21489 group - name of the stats group (string) 21490 .PP 21491 Returns the following values: 21492 .IP 21493 .nf 21494 \f[C] 21495 { 21496 \[dq]bytes\[dq]: total transferred bytes since the start of the group, 21497 \[dq]checks\[dq]: number of files checked, 21498 \[dq]deletes\[dq] : number of files deleted, 21499 \[dq]elapsedTime\[dq]: time in floating point seconds since rclone was started, 21500 \[dq]errors\[dq]: number of errors, 21501 \[dq]eta\[dq]: estimated time in seconds until the group completes, 21502 \[dq]fatalError\[dq]: boolean whether there has been at least one fatal error, 21503 \[dq]lastError\[dq]: last error string, 21504 \[dq]renames\[dq] : number of files renamed, 21505 \[dq]retryError\[dq]: boolean showing whether there has been at least one non-NoRetryError, 21506 \[dq]serverSideCopies\[dq]: number of server side copies done, 21507 \[dq]serverSideCopyBytes\[dq]: number bytes server side copied, 21508 \[dq]serverSideMoves\[dq]: number of server side moves done, 21509 \[dq]serverSideMoveBytes\[dq]: number bytes server side moved, 21510 \[dq]speed\[dq]: average speed in bytes per second since start of the group, 21511 \[dq]totalBytes\[dq]: total number of bytes in the group, 21512 \[dq]totalChecks\[dq]: total number of checks in the group, 21513 \[dq]totalTransfers\[dq]: total number of transfers in the group, 21514 \[dq]transferTime\[dq] : total time spent on running jobs, 21515 \[dq]transfers\[dq]: number of transferred files, 21516 \[dq]transferring\[dq]: an array of currently active file transfers: 21517 [ 21518 { 21519 \[dq]bytes\[dq]: total transferred bytes for this file, 21520 \[dq]eta\[dq]: estimated time in seconds until file transfer completion 21521 \[dq]name\[dq]: name of the file, 21522 \[dq]percentage\[dq]: progress of the file transfer in percent, 21523 \[dq]speed\[dq]: average speed over the whole transfer in bytes per second, 21524 \[dq]speedAvg\[dq]: current speed in bytes per second as an exponentially weighted moving average, 21525 \[dq]size\[dq]: size of the file in bytes 21526 } 21527 ], 21528 \[dq]checking\[dq]: an array of names of currently active file checks 21529 [] 21530 } 21531 \f[R] 21532 .fi 21533 .PP 21534 Values for \[dq]transferring\[dq], \[dq]checking\[dq] and 21535 \[dq]lastError\[dq] are only assigned if data is available. 21536 The value for \[dq]eta\[dq] is null if an eta cannot be determined. 21537 .SS core/stats-delete: Delete stats group. 21538 .PP 21539 This deletes entire stats group. 21540 .PP 21541 Parameters 21542 .IP \[bu] 2 21543 group - name of the stats group (string) 21544 .SS core/stats-reset: Reset stats. 21545 .PP 21546 This clears counters, errors and finished transfers for all stats or 21547 specific stats group if group is provided. 21548 .PP 21549 Parameters 21550 .IP \[bu] 2 21551 group - name of the stats group (string) 21552 .SS core/transferred: Returns stats about completed transfers. 21553 .PP 21554 This returns stats about completed transfers: 21555 .IP 21556 .nf 21557 \f[C] 21558 rclone rc core/transferred 21559 \f[R] 21560 .fi 21561 .PP 21562 If group is not provided then completed transfers for all groups will be 21563 returned. 21564 .PP 21565 Note only the last 100 completed transfers are returned. 21566 .PP 21567 Parameters 21568 .IP \[bu] 2 21569 group - name of the stats group (string) 21570 .PP 21571 Returns the following values: 21572 .IP 21573 .nf 21574 \f[C] 21575 { 21576 \[dq]transferred\[dq]: an array of completed transfers (including failed ones): 21577 [ 21578 { 21579 \[dq]name\[dq]: name of the file, 21580 \[dq]size\[dq]: size of the file in bytes, 21581 \[dq]bytes\[dq]: total transferred bytes for this file, 21582 \[dq]checked\[dq]: if the transfer is only checked (skipped, deleted), 21583 \[dq]timestamp\[dq]: integer representing millisecond unix epoch, 21584 \[dq]error\[dq]: string description of the error (empty if successful), 21585 \[dq]jobid\[dq]: id of the job that this transfer belongs to 21586 } 21587 ] 21588 } 21589 \f[R] 21590 .fi 21591 .SS core/version: Shows the current version of rclone and the go runtime. 21592 .PP 21593 This shows the current version of go and the go runtime: 21594 .IP \[bu] 2 21595 version - rclone version, e.g. 21596 \[dq]v1.53.0\[dq] 21597 .IP \[bu] 2 21598 decomposed - version number as [major, minor, patch] 21599 .IP \[bu] 2 21600 isGit - boolean - true if this was compiled from the git version 21601 .IP \[bu] 2 21602 isBeta - boolean - true if this is a beta version 21603 .IP \[bu] 2 21604 os - OS in use as according to Go 21605 .IP \[bu] 2 21606 arch - cpu architecture in use according to Go 21607 .IP \[bu] 2 21608 goVersion - version of Go runtime in use 21609 .IP \[bu] 2 21610 linking - type of rclone executable (static or dynamic) 21611 .IP \[bu] 2 21612 goTags - space separated build tags or \[dq]none\[dq] 21613 .SS debug/set-block-profile-rate: Set runtime.SetBlockProfileRate for blocking profiling. 21614 .PP 21615 SetBlockProfileRate controls the fraction of goroutine blocking events 21616 that are reported in the blocking profile. 21617 The profiler aims to sample an average of one blocking event per rate 21618 nanoseconds spent blocked. 21619 .PP 21620 To include every blocking event in the profile, pass rate = 1. 21621 To turn off profiling entirely, pass rate <= 0. 21622 .PP 21623 After calling this you can use this to see the blocking profile: 21624 .IP 21625 .nf 21626 \f[C] 21627 go tool pprof http://localhost:5572/debug/pprof/block 21628 \f[R] 21629 .fi 21630 .PP 21631 Parameters: 21632 .IP \[bu] 2 21633 rate - int 21634 .SS debug/set-gc-percent: Call runtime/debug.SetGCPercent for setting the garbage collection target percentage. 21635 .PP 21636 SetGCPercent sets the garbage collection target percentage: a collection 21637 is triggered when the ratio of freshly allocated data to live data 21638 remaining after the previous collection reaches this percentage. 21639 SetGCPercent returns the previous setting. 21640 The initial setting is the value of the GOGC environment variable at 21641 startup, or 100 if the variable is not set. 21642 .PP 21643 This setting may be effectively reduced in order to maintain a memory 21644 limit. 21645 A negative percentage effectively disables garbage collection, unless 21646 the memory limit is reached. 21647 .PP 21648 See https://pkg.go.dev/runtime/debug#SetMemoryLimit for more details. 21649 .PP 21650 Parameters: 21651 .IP \[bu] 2 21652 gc-percent - int 21653 .SS debug/set-mutex-profile-fraction: Set runtime.SetMutexProfileFraction for mutex profiling. 21654 .PP 21655 SetMutexProfileFraction controls the fraction of mutex contention events 21656 that are reported in the mutex profile. 21657 On average 1/rate events are reported. 21658 The previous rate is returned. 21659 .PP 21660 To turn off profiling entirely, pass rate 0. 21661 To just read the current rate, pass rate < 0. 21662 (For n>1 the details of sampling may change.) 21663 .PP 21664 Once this is set you can look use this to profile the mutex contention: 21665 .IP 21666 .nf 21667 \f[C] 21668 go tool pprof http://localhost:5572/debug/pprof/mutex 21669 \f[R] 21670 .fi 21671 .PP 21672 Parameters: 21673 .IP \[bu] 2 21674 rate - int 21675 .PP 21676 Results: 21677 .IP \[bu] 2 21678 previousRate - int 21679 .SS debug/set-soft-memory-limit: Call runtime/debug.SetMemoryLimit for setting a soft memory limit for the runtime. 21680 .PP 21681 SetMemoryLimit provides the runtime with a soft memory limit. 21682 .PP 21683 The runtime undertakes several processes to try to respect this memory 21684 limit, including adjustments to the frequency of garbage collections and 21685 returning memory to the underlying system more aggressively. 21686 This limit will be respected even if GOGC=off (or, if SetGCPercent(-1) 21687 is executed). 21688 .PP 21689 The input limit is provided as bytes, and includes all memory mapped, 21690 managed, and not released by the Go runtime. 21691 Notably, it does not account for space used by the Go binary and memory 21692 external to Go, such as memory managed by the underlying system on 21693 behalf of the process, or memory managed by non-Go code inside the same 21694 process. 21695 Examples of excluded memory sources include: OS kernel memory held on 21696 behalf of the process, memory allocated by C code, and memory mapped by 21697 syscall.Mmap (because it is not managed by the Go runtime). 21698 .PP 21699 A zero limit or a limit that\[aq]s lower than the amount of memory used 21700 by the Go runtime may cause the garbage collector to run nearly 21701 continuously. 21702 However, the application may still make progress. 21703 .PP 21704 The memory limit is always respected by the Go runtime, so to 21705 effectively disable this behavior, set the limit very high. 21706 math.MaxInt64 is the canonical value for disabling the limit, but values 21707 much greater than the available memory on the underlying system work 21708 just as well. 21709 .PP 21710 See https://go.dev/doc/gc-guide for a detailed guide explaining the soft 21711 memory limit in more detail, as well as a variety of common use-cases 21712 and scenarios. 21713 .PP 21714 SetMemoryLimit returns the previously set memory limit. 21715 A negative input does not adjust the limit, and allows for retrieval of 21716 the currently set memory limit. 21717 .PP 21718 Parameters: 21719 .IP \[bu] 2 21720 mem-limit - int 21721 .SS fscache/clear: Clear the Fs cache. 21722 .PP 21723 This clears the fs cache. 21724 This is where remotes created from backends are cached for a short while 21725 to make repeated rc calls more efficient. 21726 .PP 21727 If you change the parameters of a backend then you may want to call this 21728 to clear an existing remote out of the cache before re-creating it. 21729 .PP 21730 \f[B]Authentication is required for this call.\f[R] 21731 .SS fscache/entries: Returns the number of entries in the fs cache. 21732 .PP 21733 This returns the number of entries in the fs cache. 21734 .PP 21735 Returns - entries - number of items in the cache 21736 .PP 21737 \f[B]Authentication is required for this call.\f[R] 21738 .SS job/list: Lists the IDs of the running jobs 21739 .PP 21740 Parameters: None. 21741 .PP 21742 Results: 21743 .IP \[bu] 2 21744 executeId - string id of rclone executing (change after restart) 21745 .IP \[bu] 2 21746 jobids - array of integer job ids (starting at 1 on each restart) 21747 .SS job/status: Reads the status of the job ID 21748 .PP 21749 Parameters: 21750 .IP \[bu] 2 21751 jobid - id of the job (integer). 21752 .PP 21753 Results: 21754 .IP \[bu] 2 21755 finished - boolean 21756 .IP \[bu] 2 21757 duration - time in seconds that the job ran for 21758 .IP \[bu] 2 21759 endTime - time the job finished (e.g. 21760 \[dq]2018-10-26T18:50:20.528746884+01:00\[dq]) 21761 .IP \[bu] 2 21762 error - error from the job or empty string for no error 21763 .IP \[bu] 2 21764 finished - boolean whether the job has finished or not 21765 .IP \[bu] 2 21766 id - as passed in above 21767 .IP \[bu] 2 21768 startTime - time the job started (e.g. 21769 \[dq]2018-10-26T18:50:20.528336039+01:00\[dq]) 21770 .IP \[bu] 2 21771 success - boolean - true for success false otherwise 21772 .IP \[bu] 2 21773 output - output of the job as would have been returned if called 21774 synchronously 21775 .IP \[bu] 2 21776 progress - output of the progress related to the underlying job 21777 .SS job/stop: Stop the running job 21778 .PP 21779 Parameters: 21780 .IP \[bu] 2 21781 jobid - id of the job (integer). 21782 .SS job/stopgroup: Stop all running jobs in a group 21783 .PP 21784 Parameters: 21785 .IP \[bu] 2 21786 group - name of the group (string). 21787 .SS mount/listmounts: Show current mount points 21788 .PP 21789 This shows currently mounted points, which can be used for performing an 21790 unmount. 21791 .PP 21792 This takes no parameters and returns 21793 .IP \[bu] 2 21794 mountPoints: list of current mount points 21795 .PP 21796 Eg 21797 .IP 21798 .nf 21799 \f[C] 21800 rclone rc mount/listmounts 21801 \f[R] 21802 .fi 21803 .PP 21804 \f[B]Authentication is required for this call.\f[R] 21805 .SS mount/mount: Create a new mount point 21806 .PP 21807 rclone allows Linux, FreeBSD, macOS and Windows to mount any of 21808 Rclone\[aq]s cloud storage systems as a file system with FUSE. 21809 .PP 21810 If no mountType is provided, the priority is given as follows: 1. 21811 mount 2.cmount 3.mount2 21812 .PP 21813 This takes the following parameters: 21814 .IP \[bu] 2 21815 fs - a remote path to be mounted (required) 21816 .IP \[bu] 2 21817 mountPoint: valid path on the local machine (required) 21818 .IP \[bu] 2 21819 mountType: one of the values (mount, cmount, mount2) specifies the mount 21820 implementation to use 21821 .IP \[bu] 2 21822 mountOpt: a JSON object with Mount options in. 21823 .IP \[bu] 2 21824 vfsOpt: a JSON object with VFS options in. 21825 .PP 21826 Example: 21827 .IP 21828 .nf 21829 \f[C] 21830 rclone rc mount/mount fs=mydrive: mountPoint=/home/<user>/mountPoint 21831 rclone rc mount/mount fs=mydrive: mountPoint=/home/<user>/mountPoint mountType=mount 21832 rclone rc mount/mount fs=TestDrive: mountPoint=/mnt/tmp vfsOpt=\[aq]{\[dq]CacheMode\[dq]: 2}\[aq] mountOpt=\[aq]{\[dq]AllowOther\[dq]: true}\[aq] 21833 \f[R] 21834 .fi 21835 .PP 21836 The vfsOpt are as described in options/get and can be seen in the the 21837 \[dq]vfs\[dq] section when running and the mountOpt can be seen in the 21838 \[dq]mount\[dq] section: 21839 .IP 21840 .nf 21841 \f[C] 21842 rclone rc options/get 21843 \f[R] 21844 .fi 21845 .PP 21846 \f[B]Authentication is required for this call.\f[R] 21847 .SS mount/types: Show all possible mount types 21848 .PP 21849 This shows all possible mount types and returns them as a list. 21850 .PP 21851 This takes no parameters and returns 21852 .IP \[bu] 2 21853 mountTypes: list of mount types 21854 .PP 21855 The mount types are strings like \[dq]mount\[dq], \[dq]mount2\[dq], 21856 \[dq]cmount\[dq] and can be passed to mount/mount as the mountType 21857 parameter. 21858 .PP 21859 Eg 21860 .IP 21861 .nf 21862 \f[C] 21863 rclone rc mount/types 21864 \f[R] 21865 .fi 21866 .PP 21867 \f[B]Authentication is required for this call.\f[R] 21868 .SS mount/unmount: Unmount selected active mount 21869 .PP 21870 rclone allows Linux, FreeBSD, macOS and Windows to mount any of 21871 Rclone\[aq]s cloud storage systems as a file system with FUSE. 21872 .PP 21873 This takes the following parameters: 21874 .IP \[bu] 2 21875 mountPoint: valid path on the local machine where the mount was created 21876 (required) 21877 .PP 21878 Example: 21879 .IP 21880 .nf 21881 \f[C] 21882 rclone rc mount/unmount mountPoint=/home/<user>/mountPoint 21883 \f[R] 21884 .fi 21885 .PP 21886 \f[B]Authentication is required for this call.\f[R] 21887 .SS mount/unmountall: Unmount all active mounts 21888 .PP 21889 rclone allows Linux, FreeBSD, macOS and Windows to mount any of 21890 Rclone\[aq]s cloud storage systems as a file system with FUSE. 21891 .PP 21892 This takes no parameters and returns error if unmount does not succeed. 21893 .PP 21894 Eg 21895 .IP 21896 .nf 21897 \f[C] 21898 rclone rc mount/unmountall 21899 \f[R] 21900 .fi 21901 .PP 21902 \f[B]Authentication is required for this call.\f[R] 21903 .SS operations/about: Return the space used on the remote 21904 .PP 21905 This takes the following parameters: 21906 .IP \[bu] 2 21907 fs - a remote name string e.g. 21908 \[dq]drive:\[dq] 21909 .PP 21910 The result is as returned from rclone about --json 21911 .PP 21912 See the about (https://rclone.org/commands/rclone_about/) command for 21913 more information on the above. 21914 .PP 21915 \f[B]Authentication is required for this call.\f[R] 21916 .SS operations/check: check the source and destination are the same 21917 .PP 21918 Checks the files in the source and destination match. 21919 It compares sizes and hashes and logs a report of files that don\[aq]t 21920 match. 21921 It doesn\[aq]t alter the source or destination. 21922 .PP 21923 This takes the following parameters: 21924 .IP \[bu] 2 21925 srcFs - a remote name string e.g. 21926 \[dq]drive:\[dq] for the source, \[dq]/\[dq] for local filesystem 21927 .IP \[bu] 2 21928 dstFs - a remote name string e.g. 21929 \[dq]drive2:\[dq] for the destination, \[dq]/\[dq] for local filesystem 21930 .IP \[bu] 2 21931 download - check by downloading rather than with hash 21932 .IP \[bu] 2 21933 checkFileHash - treat checkFileFs:checkFileRemote as a SUM file with 21934 hashes of given type 21935 .IP \[bu] 2 21936 checkFileFs - treat checkFileFs:checkFileRemote as a SUM file with 21937 hashes of given type 21938 .IP \[bu] 2 21939 checkFileRemote - treat checkFileFs:checkFileRemote as a SUM file with 21940 hashes of given type 21941 .IP \[bu] 2 21942 oneWay - check one way only, source files must exist on remote 21943 .IP \[bu] 2 21944 combined - make a combined report of changes (default false) 21945 .IP \[bu] 2 21946 missingOnSrc - report all files missing from the source (default true) 21947 .IP \[bu] 2 21948 missingOnDst - report all files missing from the destination (default 21949 true) 21950 .IP \[bu] 2 21951 match - report all matching files (default false) 21952 .IP \[bu] 2 21953 differ - report all non-matching files (default true) 21954 .IP \[bu] 2 21955 error - report all files with errors (hashing or reading) (default true) 21956 .PP 21957 If you supply the download flag, it will download the data from both 21958 remotes and check them against each other on the fly. 21959 This can be useful for remotes that don\[aq]t support hashes or if you 21960 really want to check all the data. 21961 .PP 21962 If you supply the size-only global flag, it will only compare the sizes 21963 not the hashes as well. 21964 Use this for a quick check. 21965 .PP 21966 If you supply the checkFileHash option with a valid hash name, the 21967 checkFileFs:checkFileRemote must point to a text file in the SUM format. 21968 This treats the checksum file as the source and dstFs as the 21969 destination. 21970 Note that srcFs is not used and should not be supplied in this case. 21971 .PP 21972 Returns: 21973 .IP \[bu] 2 21974 success - true if no error, false otherwise 21975 .IP \[bu] 2 21976 status - textual summary of check, OK or text string 21977 .IP \[bu] 2 21978 hashType - hash used in check, may be missing 21979 .IP \[bu] 2 21980 combined - array of strings of combined report of changes 21981 .IP \[bu] 2 21982 missingOnSrc - array of strings of all files missing from the source 21983 .IP \[bu] 2 21984 missingOnDst - array of strings of all files missing from the 21985 destination 21986 .IP \[bu] 2 21987 match - array of strings of all matching files 21988 .IP \[bu] 2 21989 differ - array of strings of all non-matching files 21990 .IP \[bu] 2 21991 error - array of strings of all files with errors (hashing or reading) 21992 .PP 21993 \f[B]Authentication is required for this call.\f[R] 21994 .SS operations/cleanup: Remove trashed files in the remote or path 21995 .PP 21996 This takes the following parameters: 21997 .IP \[bu] 2 21998 fs - a remote name string e.g. 21999 \[dq]drive:\[dq] 22000 .PP 22001 See the cleanup (https://rclone.org/commands/rclone_cleanup/) command 22002 for more information on the above. 22003 .PP 22004 \f[B]Authentication is required for this call.\f[R] 22005 .SS operations/copyfile: Copy a file from source remote to destination remote 22006 .PP 22007 This takes the following parameters: 22008 .IP \[bu] 2 22009 srcFs - a remote name string e.g. 22010 \[dq]drive:\[dq] for the source, \[dq]/\[dq] for local filesystem 22011 .IP \[bu] 2 22012 srcRemote - a path within that remote e.g. 22013 \[dq]file.txt\[dq] for the source 22014 .IP \[bu] 2 22015 dstFs - a remote name string e.g. 22016 \[dq]drive2:\[dq] for the destination, \[dq]/\[dq] for local filesystem 22017 .IP \[bu] 2 22018 dstRemote - a path within that remote e.g. 22019 \[dq]file2.txt\[dq] for the destination 22020 .PP 22021 \f[B]Authentication is required for this call.\f[R] 22022 .SS operations/copyurl: Copy the URL to the object 22023 .PP 22024 This takes the following parameters: 22025 .IP \[bu] 2 22026 fs - a remote name string e.g. 22027 \[dq]drive:\[dq] 22028 .IP \[bu] 2 22029 remote - a path within that remote e.g. 22030 \[dq]dir\[dq] 22031 .IP \[bu] 2 22032 url - string, URL to read from 22033 .IP \[bu] 2 22034 autoFilename - boolean, set to true to retrieve destination file name 22035 from url 22036 .PP 22037 See the copyurl (https://rclone.org/commands/rclone_copyurl/) command 22038 for more information on the above. 22039 .PP 22040 \f[B]Authentication is required for this call.\f[R] 22041 .SS operations/delete: Remove files in the path 22042 .PP 22043 This takes the following parameters: 22044 .IP \[bu] 2 22045 fs - a remote name string e.g. 22046 \[dq]drive:\[dq] 22047 .PP 22048 See the delete (https://rclone.org/commands/rclone_delete/) command for 22049 more information on the above. 22050 .PP 22051 \f[B]Authentication is required for this call.\f[R] 22052 .SS operations/deletefile: Remove the single file pointed to 22053 .PP 22054 This takes the following parameters: 22055 .IP \[bu] 2 22056 fs - a remote name string e.g. 22057 \[dq]drive:\[dq] 22058 .IP \[bu] 2 22059 remote - a path within that remote e.g. 22060 \[dq]dir\[dq] 22061 .PP 22062 See the deletefile (https://rclone.org/commands/rclone_deletefile/) 22063 command for more information on the above. 22064 .PP 22065 \f[B]Authentication is required for this call.\f[R] 22066 .SS operations/fsinfo: Return information about the remote 22067 .PP 22068 This takes the following parameters: 22069 .IP \[bu] 2 22070 fs - a remote name string e.g. 22071 \[dq]drive:\[dq] 22072 .PP 22073 This returns info about the remote passed in; 22074 .IP 22075 .nf 22076 \f[C] 22077 { 22078 // optional features and whether they are available or not 22079 \[dq]Features\[dq]: { 22080 \[dq]About\[dq]: true, 22081 \[dq]BucketBased\[dq]: false, 22082 \[dq]BucketBasedRootOK\[dq]: false, 22083 \[dq]CanHaveEmptyDirectories\[dq]: true, 22084 \[dq]CaseInsensitive\[dq]: false, 22085 \[dq]ChangeNotify\[dq]: false, 22086 \[dq]CleanUp\[dq]: false, 22087 \[dq]Command\[dq]: true, 22088 \[dq]Copy\[dq]: false, 22089 \[dq]DirCacheFlush\[dq]: false, 22090 \[dq]DirMove\[dq]: true, 22091 \[dq]Disconnect\[dq]: false, 22092 \[dq]DuplicateFiles\[dq]: false, 22093 \[dq]GetTier\[dq]: false, 22094 \[dq]IsLocal\[dq]: true, 22095 \[dq]ListR\[dq]: false, 22096 \[dq]MergeDirs\[dq]: false, 22097 \[dq]MetadataInfo\[dq]: true, 22098 \[dq]Move\[dq]: true, 22099 \[dq]OpenWriterAt\[dq]: true, 22100 \[dq]PublicLink\[dq]: false, 22101 \[dq]Purge\[dq]: true, 22102 \[dq]PutStream\[dq]: true, 22103 \[dq]PutUnchecked\[dq]: false, 22104 \[dq]ReadMetadata\[dq]: true, 22105 \[dq]ReadMimeType\[dq]: false, 22106 \[dq]ServerSideAcrossConfigs\[dq]: false, 22107 \[dq]SetTier\[dq]: false, 22108 \[dq]SetWrapper\[dq]: false, 22109 \[dq]Shutdown\[dq]: false, 22110 \[dq]SlowHash\[dq]: true, 22111 \[dq]SlowModTime\[dq]: false, 22112 \[dq]UnWrap\[dq]: false, 22113 \[dq]UserInfo\[dq]: false, 22114 \[dq]UserMetadata\[dq]: true, 22115 \[dq]WrapFs\[dq]: false, 22116 \[dq]WriteMetadata\[dq]: true, 22117 \[dq]WriteMimeType\[dq]: false 22118 }, 22119 // Names of hashes available 22120 \[dq]Hashes\[dq]: [ 22121 \[dq]md5\[dq], 22122 \[dq]sha1\[dq], 22123 \[dq]whirlpool\[dq], 22124 \[dq]crc32\[dq], 22125 \[dq]sha256\[dq], 22126 \[dq]dropbox\[dq], 22127 \[dq]mailru\[dq], 22128 \[dq]quickxor\[dq] 22129 ], 22130 \[dq]Name\[dq]: \[dq]local\[dq], // Name as created 22131 \[dq]Precision\[dq]: 1, // Precision of timestamps in ns 22132 \[dq]Root\[dq]: \[dq]/\[dq], // Path as created 22133 \[dq]String\[dq]: \[dq]Local file system at /\[dq], // how the remote will appear in logs 22134 // Information about the system metadata for this backend 22135 \[dq]MetadataInfo\[dq]: { 22136 \[dq]System\[dq]: { 22137 \[dq]atime\[dq]: { 22138 \[dq]Help\[dq]: \[dq]Time of last access\[dq], 22139 \[dq]Type\[dq]: \[dq]RFC 3339\[dq], 22140 \[dq]Example\[dq]: \[dq]2006-01-02T15:04:05.999999999Z07:00\[dq] 22141 }, 22142 \[dq]btime\[dq]: { 22143 \[dq]Help\[dq]: \[dq]Time of file birth (creation)\[dq], 22144 \[dq]Type\[dq]: \[dq]RFC 3339\[dq], 22145 \[dq]Example\[dq]: \[dq]2006-01-02T15:04:05.999999999Z07:00\[dq] 22146 }, 22147 \[dq]gid\[dq]: { 22148 \[dq]Help\[dq]: \[dq]Group ID of owner\[dq], 22149 \[dq]Type\[dq]: \[dq]decimal number\[dq], 22150 \[dq]Example\[dq]: \[dq]500\[dq] 22151 }, 22152 \[dq]mode\[dq]: { 22153 \[dq]Help\[dq]: \[dq]File type and mode\[dq], 22154 \[dq]Type\[dq]: \[dq]octal, unix style\[dq], 22155 \[dq]Example\[dq]: \[dq]0100664\[dq] 22156 }, 22157 \[dq]mtime\[dq]: { 22158 \[dq]Help\[dq]: \[dq]Time of last modification\[dq], 22159 \[dq]Type\[dq]: \[dq]RFC 3339\[dq], 22160 \[dq]Example\[dq]: \[dq]2006-01-02T15:04:05.999999999Z07:00\[dq] 22161 }, 22162 \[dq]rdev\[dq]: { 22163 \[dq]Help\[dq]: \[dq]Device ID (if special file)\[dq], 22164 \[dq]Type\[dq]: \[dq]hexadecimal\[dq], 22165 \[dq]Example\[dq]: \[dq]1abc\[dq] 22166 }, 22167 \[dq]uid\[dq]: { 22168 \[dq]Help\[dq]: \[dq]User ID of owner\[dq], 22169 \[dq]Type\[dq]: \[dq]decimal number\[dq], 22170 \[dq]Example\[dq]: \[dq]500\[dq] 22171 } 22172 }, 22173 \[dq]Help\[dq]: \[dq]Textual help string\[rs]n\[dq] 22174 } 22175 } 22176 \f[R] 22177 .fi 22178 .PP 22179 This command does not have a command line equivalent so use this 22180 instead: 22181 .IP 22182 .nf 22183 \f[C] 22184 rclone rc --loopback operations/fsinfo fs=remote: 22185 \f[R] 22186 .fi 22187 .SS operations/hashsum: Produces a hashsum file for all the objects in the path. 22188 .PP 22189 Produces a hash file for all the objects in the path using the hash 22190 named. 22191 The output is in the same format as the standard md5sum/sha1sum tool. 22192 .PP 22193 This takes the following parameters: 22194 .IP \[bu] 2 22195 fs - a remote name string e.g. 22196 \[dq]drive:\[dq] for the source, \[dq]/\[dq] for local filesystem 22197 .RS 2 22198 .IP \[bu] 2 22199 this can point to a file and just that file will be returned in the 22200 listing. 22201 .RE 22202 .IP \[bu] 2 22203 hashType - type of hash to be used 22204 .IP \[bu] 2 22205 download - check by downloading rather than with hash (boolean) 22206 .IP \[bu] 2 22207 base64 - output the hashes in base64 rather than hex (boolean) 22208 .PP 22209 If you supply the download flag, it will download the data from the 22210 remote and create the hash on the fly. 22211 This can be useful for remotes that don\[aq]t support the given hash or 22212 if you really want to check all the data. 22213 .PP 22214 Note that if you wish to supply a checkfile to check hashes against the 22215 current files then you should use operations/check instead of 22216 operations/hashsum. 22217 .PP 22218 Returns: 22219 .IP \[bu] 2 22220 hashsum - array of strings of the hashes 22221 .IP \[bu] 2 22222 hashType - type of hash used 22223 .PP 22224 Example: 22225 .IP 22226 .nf 22227 \f[C] 22228 $ rclone rc --loopback operations/hashsum fs=bin hashType=MD5 download=true base64=true 22229 { 22230 \[dq]hashType\[dq]: \[dq]md5\[dq], 22231 \[dq]hashsum\[dq]: [ 22232 \[dq]WTSVLpuiXyJO_kGzJerRLg== backend-versions.sh\[dq], 22233 \[dq]v1b_OlWCJO9LtNq3EIKkNQ== bisect-go-rclone.sh\[dq], 22234 \[dq]VHbmHzHh4taXzgag8BAIKQ== bisect-rclone.sh\[dq], 22235 ] 22236 } 22237 \f[R] 22238 .fi 22239 .PP 22240 See the hashsum (https://rclone.org/commands/rclone_hashsum/) command 22241 for more information on the above. 22242 .PP 22243 \f[B]Authentication is required for this call.\f[R] 22244 .SS operations/list: List the given remote and path in JSON format 22245 .PP 22246 This takes the following parameters: 22247 .IP \[bu] 2 22248 fs - a remote name string e.g. 22249 \[dq]drive:\[dq] 22250 .IP \[bu] 2 22251 remote - a path within that remote e.g. 22252 \[dq]dir\[dq] 22253 .IP \[bu] 2 22254 opt - a dictionary of options to control the listing (optional) 22255 .RS 2 22256 .IP \[bu] 2 22257 recurse - If set recurse directories 22258 .IP \[bu] 2 22259 noModTime - If set return modification time 22260 .IP \[bu] 2 22261 showEncrypted - If set show decrypted names 22262 .IP \[bu] 2 22263 showOrigIDs - If set show the IDs for each item if known 22264 .IP \[bu] 2 22265 showHash - If set return a dictionary of hashes 22266 .IP \[bu] 2 22267 noMimeType - If set don\[aq]t show mime types 22268 .IP \[bu] 2 22269 dirsOnly - If set only show directories 22270 .IP \[bu] 2 22271 filesOnly - If set only show files 22272 .IP \[bu] 2 22273 metadata - If set return metadata of objects also 22274 .IP \[bu] 2 22275 hashTypes - array of strings of hash types to show if showHash set 22276 .RE 22277 .PP 22278 Returns: 22279 .IP \[bu] 2 22280 list 22281 .RS 2 22282 .IP \[bu] 2 22283 This is an array of objects as described in the lsjson command 22284 .RE 22285 .PP 22286 See the lsjson (https://rclone.org/commands/rclone_lsjson/) command for 22287 more information on the above and examples. 22288 .PP 22289 \f[B]Authentication is required for this call.\f[R] 22290 .SS operations/mkdir: Make a destination directory or container 22291 .PP 22292 This takes the following parameters: 22293 .IP \[bu] 2 22294 fs - a remote name string e.g. 22295 \[dq]drive:\[dq] 22296 .IP \[bu] 2 22297 remote - a path within that remote e.g. 22298 \[dq]dir\[dq] 22299 .PP 22300 See the mkdir (https://rclone.org/commands/rclone_mkdir/) command for 22301 more information on the above. 22302 .PP 22303 \f[B]Authentication is required for this call.\f[R] 22304 .SS operations/movefile: Move a file from source remote to destination remote 22305 .PP 22306 This takes the following parameters: 22307 .IP \[bu] 2 22308 srcFs - a remote name string e.g. 22309 \[dq]drive:\[dq] for the source, \[dq]/\[dq] for local filesystem 22310 .IP \[bu] 2 22311 srcRemote - a path within that remote e.g. 22312 \[dq]file.txt\[dq] for the source 22313 .IP \[bu] 2 22314 dstFs - a remote name string e.g. 22315 \[dq]drive2:\[dq] for the destination, \[dq]/\[dq] for local filesystem 22316 .IP \[bu] 2 22317 dstRemote - a path within that remote e.g. 22318 \[dq]file2.txt\[dq] for the destination 22319 .PP 22320 \f[B]Authentication is required for this call.\f[R] 22321 .SS operations/publiclink: Create or retrieve a public link to the given file or folder. 22322 .PP 22323 This takes the following parameters: 22324 .IP \[bu] 2 22325 fs - a remote name string e.g. 22326 \[dq]drive:\[dq] 22327 .IP \[bu] 2 22328 remote - a path within that remote e.g. 22329 \[dq]dir\[dq] 22330 .IP \[bu] 2 22331 unlink - boolean - if set removes the link rather than adding it 22332 (optional) 22333 .IP \[bu] 2 22334 expire - string - the expiry time of the link e.g. 22335 \[dq]1d\[dq] (optional) 22336 .PP 22337 Returns: 22338 .IP \[bu] 2 22339 url - URL of the resource 22340 .PP 22341 See the link (https://rclone.org/commands/rclone_link/) command for more 22342 information on the above. 22343 .PP 22344 \f[B]Authentication is required for this call.\f[R] 22345 .SS operations/purge: Remove a directory or container and all of its contents 22346 .PP 22347 This takes the following parameters: 22348 .IP \[bu] 2 22349 fs - a remote name string e.g. 22350 \[dq]drive:\[dq] 22351 .IP \[bu] 2 22352 remote - a path within that remote e.g. 22353 \[dq]dir\[dq] 22354 .PP 22355 See the purge (https://rclone.org/commands/rclone_purge/) command for 22356 more information on the above. 22357 .PP 22358 \f[B]Authentication is required for this call.\f[R] 22359 .SS operations/rmdir: Remove an empty directory or container 22360 .PP 22361 This takes the following parameters: 22362 .IP \[bu] 2 22363 fs - a remote name string e.g. 22364 \[dq]drive:\[dq] 22365 .IP \[bu] 2 22366 remote - a path within that remote e.g. 22367 \[dq]dir\[dq] 22368 .PP 22369 See the rmdir (https://rclone.org/commands/rclone_rmdir/) command for 22370 more information on the above. 22371 .PP 22372 \f[B]Authentication is required for this call.\f[R] 22373 .SS operations/rmdirs: Remove all the empty directories in the path 22374 .PP 22375 This takes the following parameters: 22376 .IP \[bu] 2 22377 fs - a remote name string e.g. 22378 \[dq]drive:\[dq] 22379 .IP \[bu] 2 22380 remote - a path within that remote e.g. 22381 \[dq]dir\[dq] 22382 .IP \[bu] 2 22383 leaveRoot - boolean, set to true not to delete the root 22384 .PP 22385 See the rmdirs (https://rclone.org/commands/rclone_rmdirs/) command for 22386 more information on the above. 22387 .PP 22388 \f[B]Authentication is required for this call.\f[R] 22389 .SS operations/settier: Changes storage tier or class on all files in the path 22390 .PP 22391 This takes the following parameters: 22392 .IP \[bu] 2 22393 fs - a remote name string e.g. 22394 \[dq]drive:\[dq] 22395 .PP 22396 See the settier (https://rclone.org/commands/rclone_settier/) command 22397 for more information on the above. 22398 .PP 22399 \f[B]Authentication is required for this call.\f[R] 22400 .SS operations/settierfile: Changes storage tier or class on the single file pointed to 22401 .PP 22402 This takes the following parameters: 22403 .IP \[bu] 2 22404 fs - a remote name string e.g. 22405 \[dq]drive:\[dq] 22406 .IP \[bu] 2 22407 remote - a path within that remote e.g. 22408 \[dq]dir\[dq] 22409 .PP 22410 See the settierfile (https://rclone.org/commands/rclone_settierfile/) 22411 command for more information on the above. 22412 .PP 22413 \f[B]Authentication is required for this call.\f[R] 22414 .SS operations/size: Count the number of bytes and files in remote 22415 .PP 22416 This takes the following parameters: 22417 .IP \[bu] 2 22418 fs - a remote name string e.g. 22419 \[dq]drive:path/to/dir\[dq] 22420 .PP 22421 Returns: 22422 .IP \[bu] 2 22423 count - number of files 22424 .IP \[bu] 2 22425 bytes - number of bytes in those files 22426 .PP 22427 See the size (https://rclone.org/commands/rclone_size/) command for more 22428 information on the above. 22429 .PP 22430 \f[B]Authentication is required for this call.\f[R] 22431 .SS operations/stat: Give information about the supplied file or directory 22432 .PP 22433 This takes the following parameters 22434 .IP \[bu] 2 22435 fs - a remote name string eg \[dq]drive:\[dq] 22436 .IP \[bu] 2 22437 remote - a path within that remote eg \[dq]dir\[dq] 22438 .IP \[bu] 2 22439 opt - a dictionary of options to control the listing (optional) 22440 .RS 2 22441 .IP \[bu] 2 22442 see operations/list for the options 22443 .RE 22444 .PP 22445 The result is 22446 .IP \[bu] 2 22447 item - an object as described in the lsjson command. 22448 Will be null if not found. 22449 .PP 22450 Note that if you are only interested in files then it is much more 22451 efficient to set the filesOnly flag in the options. 22452 .PP 22453 See the lsjson (https://rclone.org/commands/rclone_lsjson/) command for 22454 more information on the above and examples. 22455 .PP 22456 \f[B]Authentication is required for this call.\f[R] 22457 .SS operations/uploadfile: Upload file using multiform/form-data 22458 .PP 22459 This takes the following parameters: 22460 .IP \[bu] 2 22461 fs - a remote name string e.g. 22462 \[dq]drive:\[dq] 22463 .IP \[bu] 2 22464 remote - a path within that remote e.g. 22465 \[dq]dir\[dq] 22466 .IP \[bu] 2 22467 each part in body represents a file to be uploaded 22468 .PP 22469 See the uploadfile (https://rclone.org/commands/rclone_uploadfile/) 22470 command for more information on the above. 22471 .PP 22472 \f[B]Authentication is required for this call.\f[R] 22473 .SS options/blocks: List all the option blocks 22474 .PP 22475 Returns: - options - a list of the options block names 22476 .SS options/get: Get all the global options 22477 .PP 22478 Returns an object where keys are option block names and values are an 22479 object with the current option values in. 22480 .PP 22481 Note that these are the global options which are unaffected by use of 22482 the _config and _filter parameters. 22483 If you wish to read the parameters set in _config then use 22484 options/config and for _filter use options/filter. 22485 .PP 22486 This shows the internal names of the option within rclone which should 22487 map to the external options very easily with a few exceptions. 22488 .SS options/local: Get the currently active config for this call 22489 .PP 22490 Returns an object with the keys \[dq]config\[dq] and \[dq]filter\[dq]. 22491 The \[dq]config\[dq] key contains the local config and the 22492 \[dq]filter\[dq] key contains the local filters. 22493 .PP 22494 Note that these are the local options specific to this rc call. 22495 If _config was not supplied then they will be the global options. 22496 Likewise with \[dq]_filter\[dq]. 22497 .PP 22498 This call is mostly useful for seeing if _config and _filter passing is 22499 working. 22500 .PP 22501 This shows the internal names of the option within rclone which should 22502 map to the external options very easily with a few exceptions. 22503 .SS options/set: Set an option 22504 .PP 22505 Parameters: 22506 .IP \[bu] 2 22507 option block name containing an object with 22508 .RS 2 22509 .IP \[bu] 2 22510 key: value 22511 .RE 22512 .PP 22513 Repeated as often as required. 22514 .PP 22515 Only supply the options you wish to change. 22516 If an option is unknown it will be silently ignored. 22517 Not all options will have an effect when changed like this. 22518 .PP 22519 For example: 22520 .PP 22521 This sets DEBUG level logs (-vv) (these can be set by number or string) 22522 .IP 22523 .nf 22524 \f[C] 22525 rclone rc options/set --json \[aq]{\[dq]main\[dq]: {\[dq]LogLevel\[dq]: \[dq]DEBUG\[dq]}}\[aq] 22526 rclone rc options/set --json \[aq]{\[dq]main\[dq]: {\[dq]LogLevel\[dq]: 8}}\[aq] 22527 \f[R] 22528 .fi 22529 .PP 22530 And this sets INFO level logs (-v) 22531 .IP 22532 .nf 22533 \f[C] 22534 rclone rc options/set --json \[aq]{\[dq]main\[dq]: {\[dq]LogLevel\[dq]: \[dq]INFO\[dq]}}\[aq] 22535 \f[R] 22536 .fi 22537 .PP 22538 And this sets NOTICE level logs (normal without -v) 22539 .IP 22540 .nf 22541 \f[C] 22542 rclone rc options/set --json \[aq]{\[dq]main\[dq]: {\[dq]LogLevel\[dq]: \[dq]NOTICE\[dq]}}\[aq] 22543 \f[R] 22544 .fi 22545 .SS pluginsctl/addPlugin: Add a plugin using url 22546 .PP 22547 Used for adding a plugin to the webgui. 22548 .PP 22549 This takes the following parameters: 22550 .IP \[bu] 2 22551 url - http url of the github repo where the plugin is hosted 22552 (http://github.com/artpar/artpar-webui-react). 22553 .PP 22554 Example: 22555 .PP 22556 rclone rc pluginsctl/addPlugin 22557 .PP 22558 \f[B]Authentication is required for this call.\f[R] 22559 .SS pluginsctl/getPluginsForType: Get plugins with type criteria 22560 .PP 22561 This shows all possible plugins by a mime type. 22562 .PP 22563 This takes the following parameters: 22564 .IP \[bu] 2 22565 type - supported mime type by a loaded plugin e.g. 22566 (video/mp4, audio/mp3). 22567 .IP \[bu] 2 22568 pluginType - filter plugins based on their type e.g. 22569 (DASHBOARD, FILE_HANDLER, TERMINAL). 22570 .PP 22571 Returns: 22572 .IP \[bu] 2 22573 loadedPlugins - list of current production plugins. 22574 .IP \[bu] 2 22575 testPlugins - list of temporarily loaded development plugins, usually 22576 running on a different server. 22577 .PP 22578 Example: 22579 .PP 22580 rclone rc pluginsctl/getPluginsForType type=video/mp4 22581 .PP 22582 \f[B]Authentication is required for this call.\f[R] 22583 .SS pluginsctl/listPlugins: Get the list of currently loaded plugins 22584 .PP 22585 This allows you to get the currently enabled plugins and their details. 22586 .PP 22587 This takes no parameters and returns: 22588 .IP \[bu] 2 22589 loadedPlugins - list of current production plugins. 22590 .IP \[bu] 2 22591 testPlugins - list of temporarily loaded development plugins, usually 22592 running on a different server. 22593 .PP 22594 E.g. 22595 .PP 22596 rclone rc pluginsctl/listPlugins 22597 .PP 22598 \f[B]Authentication is required for this call.\f[R] 22599 .SS pluginsctl/listTestPlugins: Show currently loaded test plugins 22600 .PP 22601 Allows listing of test plugins with the rclone.test set to true in 22602 package.json of the plugin. 22603 .PP 22604 This takes no parameters and returns: 22605 .IP \[bu] 2 22606 loadedTestPlugins - list of currently available test plugins. 22607 .PP 22608 E.g. 22609 .IP 22610 .nf 22611 \f[C] 22612 rclone rc pluginsctl/listTestPlugins 22613 \f[R] 22614 .fi 22615 .PP 22616 \f[B]Authentication is required for this call.\f[R] 22617 .SS pluginsctl/removePlugin: Remove a loaded plugin 22618 .PP 22619 This allows you to remove a plugin using it\[aq]s name. 22620 .PP 22621 This takes parameters: 22622 .IP \[bu] 2 22623 name - name of the plugin in the format 22624 \f[C]author\f[R]/\f[C]plugin_name\f[R]. 22625 .PP 22626 E.g. 22627 .PP 22628 rclone rc pluginsctl/removePlugin name=rclone/video-plugin 22629 .PP 22630 \f[B]Authentication is required for this call.\f[R] 22631 .SS pluginsctl/removeTestPlugin: Remove a test plugin 22632 .PP 22633 This allows you to remove a plugin using it\[aq]s name. 22634 .PP 22635 This takes the following parameters: 22636 .IP \[bu] 2 22637 name - name of the plugin in the format 22638 \f[C]author\f[R]/\f[C]plugin_name\f[R]. 22639 .PP 22640 Example: 22641 .IP 22642 .nf 22643 \f[C] 22644 rclone rc pluginsctl/removeTestPlugin name=artpar/artpar-webui-react 22645 \f[R] 22646 .fi 22647 .PP 22648 \f[B]Authentication is required for this call.\f[R] 22649 .SS rc/error: This returns an error 22650 .PP 22651 This returns an error with the input as part of its error string. 22652 Useful for testing error handling. 22653 .SS rc/list: List all the registered remote control commands 22654 .PP 22655 This lists all the registered remote control commands as a JSON map in 22656 the commands response. 22657 .SS rc/noop: Echo the input to the output parameters 22658 .PP 22659 This echoes the input parameters to the output parameters for testing 22660 purposes. 22661 It can be used to check that rclone is still alive and to check that 22662 parameter passing is working properly. 22663 .SS rc/noopauth: Echo the input to the output parameters requiring auth 22664 .PP 22665 This echoes the input parameters to the output parameters for testing 22666 purposes. 22667 It can be used to check that rclone is still alive and to check that 22668 parameter passing is working properly. 22669 .PP 22670 \f[B]Authentication is required for this call.\f[R] 22671 .SS sync/bisync: Perform bidirectional synchronization between two paths. 22672 .PP 22673 This takes the following parameters 22674 .IP \[bu] 2 22675 path1 - a remote directory string e.g. 22676 \f[C]drive:path1\f[R] 22677 .IP \[bu] 2 22678 path2 - a remote directory string e.g. 22679 \f[C]drive:path2\f[R] 22680 .IP \[bu] 2 22681 dryRun - dry-run mode 22682 .IP \[bu] 2 22683 resync - performs the resync run 22684 .IP \[bu] 2 22685 checkAccess - abort if RCLONE_TEST files are not found on both 22686 filesystems 22687 .IP \[bu] 2 22688 checkFilename - file name for checkAccess (default: RCLONE_TEST) 22689 .IP \[bu] 2 22690 maxDelete - abort sync if percentage of deleted files is above this 22691 threshold (default: 50) 22692 .IP \[bu] 2 22693 force - Bypass maxDelete safety check and run the sync 22694 .IP \[bu] 2 22695 checkSync - \f[C]true\f[R] by default, \f[C]false\f[R] disables 22696 comparison of final listings, \f[C]only\f[R] will skip sync, only 22697 compare listings from the last run 22698 .IP \[bu] 2 22699 createEmptySrcDirs - Sync creation and deletion of empty directories. 22700 (Not compatible with --remove-empty-dirs) 22701 .IP \[bu] 2 22702 removeEmptyDirs - remove empty directories at the final cleanup step 22703 .IP \[bu] 2 22704 filtersFile - read filtering patterns from a file 22705 .IP \[bu] 2 22706 ignoreListingChecksum - Do not use checksums for listings 22707 .IP \[bu] 2 22708 resilient - Allow future runs to retry after certain less-serious 22709 errors, instead of requiring resync. 22710 Use at your own risk! 22711 .IP \[bu] 2 22712 workdir - server directory for history files (default: 22713 \f[C]\[ti]/.cache/rclone/bisync\f[R]) 22714 .IP \[bu] 2 22715 backupdir1 - --backup-dir for Path1. 22716 Must be a non-overlapping path on the same remote. 22717 .IP \[bu] 2 22718 backupdir2 - --backup-dir for Path2. 22719 Must be a non-overlapping path on the same remote. 22720 .IP \[bu] 2 22721 noCleanup - retain working files 22722 .PP 22723 See bisync command help (https://rclone.org/commands/rclone_bisync/) and 22724 full bisync description (https://rclone.org/bisync/) for more 22725 information. 22726 .PP 22727 \f[B]Authentication is required for this call.\f[R] 22728 .SS sync/copy: copy a directory from source remote to destination remote 22729 .PP 22730 This takes the following parameters: 22731 .IP \[bu] 2 22732 srcFs - a remote name string e.g. 22733 \[dq]drive:src\[dq] for the source 22734 .IP \[bu] 2 22735 dstFs - a remote name string e.g. 22736 \[dq]drive:dst\[dq] for the destination 22737 .IP \[bu] 2 22738 createEmptySrcDirs - create empty src directories on destination if set 22739 .PP 22740 See the copy (https://rclone.org/commands/rclone_copy/) command for more 22741 information on the above. 22742 .PP 22743 \f[B]Authentication is required for this call.\f[R] 22744 .SS sync/move: move a directory from source remote to destination remote 22745 .PP 22746 This takes the following parameters: 22747 .IP \[bu] 2 22748 srcFs - a remote name string e.g. 22749 \[dq]drive:src\[dq] for the source 22750 .IP \[bu] 2 22751 dstFs - a remote name string e.g. 22752 \[dq]drive:dst\[dq] for the destination 22753 .IP \[bu] 2 22754 createEmptySrcDirs - create empty src directories on destination if set 22755 .IP \[bu] 2 22756 deleteEmptySrcDirs - delete empty src directories if set 22757 .PP 22758 See the move (https://rclone.org/commands/rclone_move/) command for more 22759 information on the above. 22760 .PP 22761 \f[B]Authentication is required for this call.\f[R] 22762 .SS sync/sync: sync a directory from source remote to destination remote 22763 .PP 22764 This takes the following parameters: 22765 .IP \[bu] 2 22766 srcFs - a remote name string e.g. 22767 \[dq]drive:src\[dq] for the source 22768 .IP \[bu] 2 22769 dstFs - a remote name string e.g. 22770 \[dq]drive:dst\[dq] for the destination 22771 .IP \[bu] 2 22772 createEmptySrcDirs - create empty src directories on destination if set 22773 .PP 22774 See the sync (https://rclone.org/commands/rclone_sync/) command for more 22775 information on the above. 22776 .PP 22777 \f[B]Authentication is required for this call.\f[R] 22778 .SS vfs/forget: Forget files or directories in the directory cache. 22779 .PP 22780 This forgets the paths in the directory cache causing them to be re-read 22781 from the remote when needed. 22782 .PP 22783 If no paths are passed in then it will forget all the paths in the 22784 directory cache. 22785 .IP 22786 .nf 22787 \f[C] 22788 rclone rc vfs/forget 22789 \f[R] 22790 .fi 22791 .PP 22792 Otherwise pass files or dirs in as file=path or dir=path. 22793 Any parameter key starting with file will forget that file and any 22794 starting with dir will forget that dir, e.g. 22795 .IP 22796 .nf 22797 \f[C] 22798 rclone rc vfs/forget file=hello file2=goodbye dir=home/junk 22799 \f[R] 22800 .fi 22801 .PP 22802 This command takes an \[dq]fs\[dq] parameter. 22803 If this parameter is not supplied and if there is only one VFS in use 22804 then that VFS will be used. 22805 If there is more than one VFS in use then the \[dq]fs\[dq] parameter 22806 must be supplied. 22807 .SS vfs/list: List active VFSes. 22808 .PP 22809 This lists the active VFSes. 22810 .PP 22811 It returns a list under the key \[dq]vfses\[dq] where the values are the 22812 VFS names that could be passed to the other VFS commands in the 22813 \[dq]fs\[dq] parameter. 22814 .SS vfs/poll-interval: Get the status or update the value of the poll-interval option. 22815 .PP 22816 Without any parameter given this returns the current status of the 22817 poll-interval setting. 22818 .PP 22819 When the interval=duration parameter is set, the poll-interval value is 22820 updated and the polling function is notified. 22821 Setting interval=0 disables poll-interval. 22822 .IP 22823 .nf 22824 \f[C] 22825 rclone rc vfs/poll-interval interval=5m 22826 \f[R] 22827 .fi 22828 .PP 22829 The timeout=duration parameter can be used to specify a time to wait for 22830 the current poll function to apply the new value. 22831 If timeout is less or equal 0, which is the default, wait indefinitely. 22832 .PP 22833 The new poll-interval value will only be active when the timeout is not 22834 reached. 22835 .PP 22836 If poll-interval is updated or disabled temporarily, some changes might 22837 not get picked up by the polling function, depending on the used remote. 22838 .PP 22839 This command takes an \[dq]fs\[dq] parameter. 22840 If this parameter is not supplied and if there is only one VFS in use 22841 then that VFS will be used. 22842 If there is more than one VFS in use then the \[dq]fs\[dq] parameter 22843 must be supplied. 22844 .SS vfs/refresh: Refresh the directory cache. 22845 .PP 22846 This reads the directories for the specified paths and freshens the 22847 directory cache. 22848 .PP 22849 If no paths are passed in then it will refresh the root directory. 22850 .IP 22851 .nf 22852 \f[C] 22853 rclone rc vfs/refresh 22854 \f[R] 22855 .fi 22856 .PP 22857 Otherwise pass directories in as dir=path. 22858 Any parameter key starting with dir will refresh that directory, e.g. 22859 .IP 22860 .nf 22861 \f[C] 22862 rclone rc vfs/refresh dir=home/junk dir2=data/misc 22863 \f[R] 22864 .fi 22865 .PP 22866 If the parameter recursive=true is given the whole directory tree will 22867 get refreshed. 22868 This refresh will use --fast-list if enabled. 22869 .PP 22870 This command takes an \[dq]fs\[dq] parameter. 22871 If this parameter is not supplied and if there is only one VFS in use 22872 then that VFS will be used. 22873 If there is more than one VFS in use then the \[dq]fs\[dq] parameter 22874 must be supplied. 22875 .SS vfs/stats: Stats for a VFS. 22876 .PP 22877 This returns stats for the selected VFS. 22878 .IP 22879 .nf 22880 \f[C] 22881 { 22882 // Status of the disk cache - only present if --vfs-cache-mode > off 22883 \[dq]diskCache\[dq]: { 22884 \[dq]bytesUsed\[dq]: 0, 22885 \[dq]erroredFiles\[dq]: 0, 22886 \[dq]files\[dq]: 0, 22887 \[dq]hashType\[dq]: 1, 22888 \[dq]outOfSpace\[dq]: false, 22889 \[dq]path\[dq]: \[dq]/home/user/.cache/rclone/vfs/local/mnt/a\[dq], 22890 \[dq]pathMeta\[dq]: \[dq]/home/user/.cache/rclone/vfsMeta/local/mnt/a\[dq], 22891 \[dq]uploadsInProgress\[dq]: 0, 22892 \[dq]uploadsQueued\[dq]: 0 22893 }, 22894 \[dq]fs\[dq]: \[dq]/mnt/a\[dq], 22895 \[dq]inUse\[dq]: 1, 22896 // Status of the in memory metadata cache 22897 \[dq]metadataCache\[dq]: { 22898 \[dq]dirs\[dq]: 1, 22899 \[dq]files\[dq]: 0 22900 }, 22901 // Options as returned by options/get 22902 \[dq]opt\[dq]: { 22903 \[dq]CacheMaxAge\[dq]: 3600000000000, 22904 // ... 22905 \[dq]WriteWait\[dq]: 1000000000 22906 } 22907 } 22908 \f[R] 22909 .fi 22910 .PP 22911 This command takes an \[dq]fs\[dq] parameter. 22912 If this parameter is not supplied and if there is only one VFS in use 22913 then that VFS will be used. 22914 If there is more than one VFS in use then the \[dq]fs\[dq] parameter 22915 must be supplied. 22916 .SS Accessing the remote control via HTTP 22917 .PP 22918 Rclone implements a simple HTTP based protocol. 22919 .PP 22920 Each endpoint takes an JSON object and returns a JSON object or an 22921 error. 22922 The JSON objects are essentially a map of string names to values. 22923 .PP 22924 All calls must made using POST. 22925 .PP 22926 The input objects can be supplied using URL parameters, POST parameters 22927 or by supplying \[dq]Content-Type: application/json\[dq] and a JSON blob 22928 in the body. 22929 There are examples of these below using \f[C]curl\f[R]. 22930 .PP 22931 The response will be a JSON blob in the body of the response. 22932 This is formatted to be reasonably human-readable. 22933 .SS Error returns 22934 .PP 22935 If an error occurs then there will be an HTTP error status (e.g. 22936 500) and the body of the response will contain a JSON encoded error 22937 object, e.g. 22938 .IP 22939 .nf 22940 \f[C] 22941 { 22942 \[dq]error\[dq]: \[dq]Expecting string value for key \[rs]\[dq]remote\[rs]\[dq] (was float64)\[dq], 22943 \[dq]input\[dq]: { 22944 \[dq]fs\[dq]: \[dq]/tmp\[dq], 22945 \[dq]remote\[dq]: 3 22946 }, 22947 \[dq]status\[dq]: 400 22948 \[dq]path\[dq]: \[dq]operations/rmdir\[dq], 22949 } 22950 \f[R] 22951 .fi 22952 .PP 22953 The keys in the error response are - error - error string - input - the 22954 input parameters to the call - status - the HTTP status code - path - 22955 the path of the call 22956 .SS CORS 22957 .PP 22958 The sever implements basic CORS support and allows all origins for that. 22959 The response to a preflight OPTIONS request will echo the requested 22960 \[dq]Access-Control-Request-Headers\[dq] back. 22961 .SS Using POST with URL parameters only 22962 .IP 22963 .nf 22964 \f[C] 22965 curl -X POST \[aq]http://localhost:5572/rc/noop?potato=1&sausage=2\[aq] 22966 \f[R] 22967 .fi 22968 .PP 22969 Response 22970 .IP 22971 .nf 22972 \f[C] 22973 { 22974 \[dq]potato\[dq]: \[dq]1\[dq], 22975 \[dq]sausage\[dq]: \[dq]2\[dq] 22976 } 22977 \f[R] 22978 .fi 22979 .PP 22980 Here is what an error response looks like: 22981 .IP 22982 .nf 22983 \f[C] 22984 curl -X POST \[aq]http://localhost:5572/rc/error?potato=1&sausage=2\[aq] 22985 \f[R] 22986 .fi 22987 .IP 22988 .nf 22989 \f[C] 22990 { 22991 \[dq]error\[dq]: \[dq]arbitrary error on input map[potato:1 sausage:2]\[dq], 22992 \[dq]input\[dq]: { 22993 \[dq]potato\[dq]: \[dq]1\[dq], 22994 \[dq]sausage\[dq]: \[dq]2\[dq] 22995 } 22996 } 22997 \f[R] 22998 .fi 22999 .PP 23000 Note that curl doesn\[aq]t return errors to the shell unless you use the 23001 \f[C]-f\f[R] option 23002 .IP 23003 .nf 23004 \f[C] 23005 $ curl -f -X POST \[aq]http://localhost:5572/rc/error?potato=1&sausage=2\[aq] 23006 curl: (22) The requested URL returned error: 400 Bad Request 23007 $ echo $? 23008 22 23009 \f[R] 23010 .fi 23011 .SS Using POST with a form 23012 .IP 23013 .nf 23014 \f[C] 23015 curl --data \[dq]potato=1\[dq] --data \[dq]sausage=2\[dq] http://localhost:5572/rc/noop 23016 \f[R] 23017 .fi 23018 .PP 23019 Response 23020 .IP 23021 .nf 23022 \f[C] 23023 { 23024 \[dq]potato\[dq]: \[dq]1\[dq], 23025 \[dq]sausage\[dq]: \[dq]2\[dq] 23026 } 23027 \f[R] 23028 .fi 23029 .PP 23030 Note that you can combine these with URL parameters too with the POST 23031 parameters taking precedence. 23032 .IP 23033 .nf 23034 \f[C] 23035 curl --data \[dq]potato=1\[dq] --data \[dq]sausage=2\[dq] \[dq]http://localhost:5572/rc/noop?rutabaga=3&sausage=4\[dq] 23036 \f[R] 23037 .fi 23038 .PP 23039 Response 23040 .IP 23041 .nf 23042 \f[C] 23043 { 23044 \[dq]potato\[dq]: \[dq]1\[dq], 23045 \[dq]rutabaga\[dq]: \[dq]3\[dq], 23046 \[dq]sausage\[dq]: \[dq]4\[dq] 23047 } 23048 \f[R] 23049 .fi 23050 .SS Using POST with a JSON blob 23051 .IP 23052 .nf 23053 \f[C] 23054 curl -H \[dq]Content-Type: application/json\[dq] -X POST -d \[aq]{\[dq]potato\[dq]:2,\[dq]sausage\[dq]:1}\[aq] http://localhost:5572/rc/noop 23055 \f[R] 23056 .fi 23057 .PP 23058 response 23059 .IP 23060 .nf 23061 \f[C] 23062 { 23063 \[dq]password\[dq]: \[dq]xyz\[dq], 23064 \[dq]username\[dq]: \[dq]xyz\[dq] 23065 } 23066 \f[R] 23067 .fi 23068 .PP 23069 This can be combined with URL parameters too if required. 23070 The JSON blob takes precedence. 23071 .IP 23072 .nf 23073 \f[C] 23074 curl -H \[dq]Content-Type: application/json\[dq] -X POST -d \[aq]{\[dq]potato\[dq]:2,\[dq]sausage\[dq]:1}\[aq] \[aq]http://localhost:5572/rc/noop?rutabaga=3&potato=4\[aq] 23075 \f[R] 23076 .fi 23077 .IP 23078 .nf 23079 \f[C] 23080 { 23081 \[dq]potato\[dq]: 2, 23082 \[dq]rutabaga\[dq]: \[dq]3\[dq], 23083 \[dq]sausage\[dq]: 1 23084 } 23085 \f[R] 23086 .fi 23087 .SS Debugging rclone with pprof 23088 .PP 23089 If you use the \f[C]--rc\f[R] flag this will also enable the use of the 23090 go profiling tools on the same port. 23091 .PP 23092 To use these, first install go (https://golang.org/doc/install). 23093 .SS Debugging memory use 23094 .PP 23095 To profile rclone\[aq]s memory use you can run: 23096 .IP 23097 .nf 23098 \f[C] 23099 go tool pprof -web http://localhost:5572/debug/pprof/heap 23100 \f[R] 23101 .fi 23102 .PP 23103 This should open a page in your browser showing what is using what 23104 memory. 23105 .PP 23106 You can also use the \f[C]-text\f[R] flag to produce a textual summary 23107 .IP 23108 .nf 23109 \f[C] 23110 $ go tool pprof -text http://localhost:5572/debug/pprof/heap 23111 Showing nodes accounting for 1537.03kB, 100% of 1537.03kB total 23112 flat flat% sum% cum cum% 23113 1024.03kB 66.62% 66.62% 1024.03kB 66.62% github.com/artpar/artpar/vendor/golang.org/x/net/http2/hpack.addDecoderNode 23114 513kB 33.38% 100% 513kB 33.38% net/http.newBufioWriterSize 23115 0 0% 100% 1024.03kB 66.62% github.com/artpar/artpar/cmd/all.init 23116 0 0% 100% 1024.03kB 66.62% github.com/artpar/artpar/cmd/serve.init 23117 0 0% 100% 1024.03kB 66.62% github.com/artpar/artpar/cmd/serve/restic.init 23118 0 0% 100% 1024.03kB 66.62% github.com/artpar/artpar/vendor/golang.org/x/net/http2.init 23119 0 0% 100% 1024.03kB 66.62% github.com/artpar/artpar/vendor/golang.org/x/net/http2/hpack.init 23120 0 0% 100% 1024.03kB 66.62% github.com/artpar/artpar/vendor/golang.org/x/net/http2/hpack.init.0 23121 0 0% 100% 1024.03kB 66.62% main.init 23122 0 0% 100% 513kB 33.38% net/http.(*conn).readRequest 23123 0 0% 100% 513kB 33.38% net/http.(*conn).serve 23124 0 0% 100% 1024.03kB 66.62% runtime.main 23125 \f[R] 23126 .fi 23127 .SS Debugging go routine leaks 23128 .PP 23129 Memory leaks are most often caused by go routine leaks keeping memory 23130 alive which should have been garbage collected. 23131 .PP 23132 See all active go routines using 23133 .IP 23134 .nf 23135 \f[C] 23136 curl http://localhost:5572/debug/pprof/goroutine?debug=1 23137 \f[R] 23138 .fi 23139 .PP 23140 Or go to http://localhost:5572/debug/pprof/goroutine?debug=1 in your 23141 browser. 23142 .SS Other profiles to look at 23143 .PP 23144 You can see a summary of profiles available at 23145 http://localhost:5572/debug/pprof/ 23146 .PP 23147 Here is how to use some of them: 23148 .IP \[bu] 2 23149 Memory: \f[C]go tool pprof http://localhost:5572/debug/pprof/heap\f[R] 23150 .IP \[bu] 2 23151 Go routines: 23152 \f[C]curl http://localhost:5572/debug/pprof/goroutine?debug=1\f[R] 23153 .IP \[bu] 2 23154 30-second CPU profile: 23155 \f[C]go tool pprof http://localhost:5572/debug/pprof/profile\f[R] 23156 .IP \[bu] 2 23157 5-second execution trace: 23158 \f[C]wget http://localhost:5572/debug/pprof/trace?seconds=5\f[R] 23159 .IP \[bu] 2 23160 Goroutine blocking profile 23161 .RS 2 23162 .IP \[bu] 2 23163 Enable first with: 23164 \f[C]rclone rc debug/set-block-profile-rate rate=1\f[R] (docs) 23165 .IP \[bu] 2 23166 \f[C]go tool pprof http://localhost:5572/debug/pprof/block\f[R] 23167 .RE 23168 .IP \[bu] 2 23169 Contended mutexes: 23170 .RS 2 23171 .IP \[bu] 2 23172 Enable first with: 23173 \f[C]rclone rc debug/set-mutex-profile-fraction rate=1\f[R] (docs) 23174 .IP \[bu] 2 23175 \f[C]go tool pprof http://localhost:5572/debug/pprof/mutex\f[R] 23176 .RE 23177 .PP 23178 See the net/http/pprof docs (https://golang.org/pkg/net/http/pprof/) for 23179 more info on how to use the profiling and for a general overview see the 23180 Go team\[aq]s blog post on profiling go 23181 programs (https://blog.golang.org/profiling-go-programs). 23182 .PP 23183 The profiling hook is zero overhead unless it is 23184 used (https://stackoverflow.com/q/26545159/164234). 23185 .SH Overview of cloud storage systems 23186 .PP 23187 Each cloud storage system is slightly different. 23188 Rclone attempts to provide a unified interface to them, but some 23189 underlying differences show through. 23190 .SS Features 23191 .PP 23192 Here is an overview of the major features of each cloud storage system. 23193 .PP 23194 .TS 23195 tab(@); 23196 l c c c c c c. 23197 T{ 23198 Name 23199 T}@T{ 23200 Hash 23201 T}@T{ 23202 ModTime 23203 T}@T{ 23204 Case Insensitive 23205 T}@T{ 23206 Duplicate Files 23207 T}@T{ 23208 MIME Type 23209 T}@T{ 23210 Metadata 23211 T} 23212 _ 23213 T{ 23214 1Fichier 23215 T}@T{ 23216 Whirlpool 23217 T}@T{ 23218 - 23219 T}@T{ 23220 No 23221 T}@T{ 23222 Yes 23223 T}@T{ 23224 R 23225 T}@T{ 23226 - 23227 T} 23228 T{ 23229 Akamai Netstorage 23230 T}@T{ 23231 MD5, SHA256 23232 T}@T{ 23233 R/W 23234 T}@T{ 23235 No 23236 T}@T{ 23237 No 23238 T}@T{ 23239 R 23240 T}@T{ 23241 - 23242 T} 23243 T{ 23244 Amazon S3 (or S3 compatible) 23245 T}@T{ 23246 MD5 23247 T}@T{ 23248 R/W 23249 T}@T{ 23250 No 23251 T}@T{ 23252 No 23253 T}@T{ 23254 R/W 23255 T}@T{ 23256 RWU 23257 T} 23258 T{ 23259 Backblaze B2 23260 T}@T{ 23261 SHA1 23262 T}@T{ 23263 R/W 23264 T}@T{ 23265 No 23266 T}@T{ 23267 No 23268 T}@T{ 23269 R/W 23270 T}@T{ 23271 - 23272 T} 23273 T{ 23274 Box 23275 T}@T{ 23276 SHA1 23277 T}@T{ 23278 R/W 23279 T}@T{ 23280 Yes 23281 T}@T{ 23282 No 23283 T}@T{ 23284 - 23285 T}@T{ 23286 - 23287 T} 23288 T{ 23289 Citrix ShareFile 23290 T}@T{ 23291 MD5 23292 T}@T{ 23293 R/W 23294 T}@T{ 23295 Yes 23296 T}@T{ 23297 No 23298 T}@T{ 23299 - 23300 T}@T{ 23301 - 23302 T} 23303 T{ 23304 Dropbox 23305 T}@T{ 23306 DBHASH \[S1] 23307 T}@T{ 23308 R 23309 T}@T{ 23310 Yes 23311 T}@T{ 23312 No 23313 T}@T{ 23314 - 23315 T}@T{ 23316 - 23317 T} 23318 T{ 23319 Enterprise File Fabric 23320 T}@T{ 23321 - 23322 T}@T{ 23323 R/W 23324 T}@T{ 23325 Yes 23326 T}@T{ 23327 No 23328 T}@T{ 23329 R/W 23330 T}@T{ 23331 - 23332 T} 23333 T{ 23334 FTP 23335 T}@T{ 23336 - 23337 T}@T{ 23338 R/W \[S1]\[u2070] 23339 T}@T{ 23340 No 23341 T}@T{ 23342 No 23343 T}@T{ 23344 - 23345 T}@T{ 23346 - 23347 T} 23348 T{ 23349 Google Cloud Storage 23350 T}@T{ 23351 MD5 23352 T}@T{ 23353 R/W 23354 T}@T{ 23355 No 23356 T}@T{ 23357 No 23358 T}@T{ 23359 R/W 23360 T}@T{ 23361 - 23362 T} 23363 T{ 23364 Google Drive 23365 T}@T{ 23366 MD5, SHA1, SHA256 23367 T}@T{ 23368 DR/W 23369 T}@T{ 23370 No 23371 T}@T{ 23372 Yes 23373 T}@T{ 23374 R/W 23375 T}@T{ 23376 DRWU 23377 T} 23378 T{ 23379 Google Photos 23380 T}@T{ 23381 - 23382 T}@T{ 23383 - 23384 T}@T{ 23385 No 23386 T}@T{ 23387 Yes 23388 T}@T{ 23389 R 23390 T}@T{ 23391 - 23392 T} 23393 T{ 23394 HDFS 23395 T}@T{ 23396 - 23397 T}@T{ 23398 R/W 23399 T}@T{ 23400 No 23401 T}@T{ 23402 No 23403 T}@T{ 23404 - 23405 T}@T{ 23406 - 23407 T} 23408 T{ 23409 HiDrive 23410 T}@T{ 23411 HiDrive \[S1]\[S2] 23412 T}@T{ 23413 R/W 23414 T}@T{ 23415 No 23416 T}@T{ 23417 No 23418 T}@T{ 23419 - 23420 T}@T{ 23421 - 23422 T} 23423 T{ 23424 HTTP 23425 T}@T{ 23426 - 23427 T}@T{ 23428 R 23429 T}@T{ 23430 No 23431 T}@T{ 23432 No 23433 T}@T{ 23434 R 23435 T}@T{ 23436 - 23437 T} 23438 T{ 23439 Internet Archive 23440 T}@T{ 23441 MD5, SHA1, CRC32 23442 T}@T{ 23443 R/W \[S1]\[S1] 23444 T}@T{ 23445 No 23446 T}@T{ 23447 No 23448 T}@T{ 23449 - 23450 T}@T{ 23451 RWU 23452 T} 23453 T{ 23454 Jottacloud 23455 T}@T{ 23456 MD5 23457 T}@T{ 23458 R/W 23459 T}@T{ 23460 Yes 23461 T}@T{ 23462 No 23463 T}@T{ 23464 R 23465 T}@T{ 23466 RW 23467 T} 23468 T{ 23469 Koofr 23470 T}@T{ 23471 MD5 23472 T}@T{ 23473 - 23474 T}@T{ 23475 Yes 23476 T}@T{ 23477 No 23478 T}@T{ 23479 - 23480 T}@T{ 23481 - 23482 T} 23483 T{ 23484 Linkbox 23485 T}@T{ 23486 - 23487 T}@T{ 23488 R 23489 T}@T{ 23490 No 23491 T}@T{ 23492 No 23493 T}@T{ 23494 - 23495 T}@T{ 23496 - 23497 T} 23498 T{ 23499 Mail.ru Cloud 23500 T}@T{ 23501 Mailru \[u2076] 23502 T}@T{ 23503 R/W 23504 T}@T{ 23505 Yes 23506 T}@T{ 23507 No 23508 T}@T{ 23509 - 23510 T}@T{ 23511 - 23512 T} 23513 T{ 23514 Mega 23515 T}@T{ 23516 - 23517 T}@T{ 23518 - 23519 T}@T{ 23520 No 23521 T}@T{ 23522 Yes 23523 T}@T{ 23524 - 23525 T}@T{ 23526 - 23527 T} 23528 T{ 23529 Memory 23530 T}@T{ 23531 MD5 23532 T}@T{ 23533 R/W 23534 T}@T{ 23535 No 23536 T}@T{ 23537 No 23538 T}@T{ 23539 - 23540 T}@T{ 23541 - 23542 T} 23543 T{ 23544 Microsoft Azure Blob Storage 23545 T}@T{ 23546 MD5 23547 T}@T{ 23548 R/W 23549 T}@T{ 23550 No 23551 T}@T{ 23552 No 23553 T}@T{ 23554 R/W 23555 T}@T{ 23556 - 23557 T} 23558 T{ 23559 Microsoft Azure Files Storage 23560 T}@T{ 23561 MD5 23562 T}@T{ 23563 R/W 23564 T}@T{ 23565 Yes 23566 T}@T{ 23567 No 23568 T}@T{ 23569 R/W 23570 T}@T{ 23571 - 23572 T} 23573 T{ 23574 Microsoft OneDrive 23575 T}@T{ 23576 QuickXorHash \[u2075] 23577 T}@T{ 23578 DR/W 23579 T}@T{ 23580 Yes 23581 T}@T{ 23582 No 23583 T}@T{ 23584 R 23585 T}@T{ 23586 DRW 23587 T} 23588 T{ 23589 OpenDrive 23590 T}@T{ 23591 MD5 23592 T}@T{ 23593 R/W 23594 T}@T{ 23595 Yes 23596 T}@T{ 23597 Partial \[u2078] 23598 T}@T{ 23599 - 23600 T}@T{ 23601 - 23602 T} 23603 T{ 23604 OpenStack Swift 23605 T}@T{ 23606 MD5 23607 T}@T{ 23608 R/W 23609 T}@T{ 23610 No 23611 T}@T{ 23612 No 23613 T}@T{ 23614 R/W 23615 T}@T{ 23616 - 23617 T} 23618 T{ 23619 Oracle Object Storage 23620 T}@T{ 23621 MD5 23622 T}@T{ 23623 R/W 23624 T}@T{ 23625 No 23626 T}@T{ 23627 No 23628 T}@T{ 23629 R/W 23630 T}@T{ 23631 - 23632 T} 23633 T{ 23634 pCloud 23635 T}@T{ 23636 MD5, SHA1 \[u2077] 23637 T}@T{ 23638 R 23639 T}@T{ 23640 No 23641 T}@T{ 23642 No 23643 T}@T{ 23644 W 23645 T}@T{ 23646 - 23647 T} 23648 T{ 23649 PikPak 23650 T}@T{ 23651 MD5 23652 T}@T{ 23653 R 23654 T}@T{ 23655 No 23656 T}@T{ 23657 No 23658 T}@T{ 23659 R 23660 T}@T{ 23661 - 23662 T} 23663 T{ 23664 premiumize.me 23665 T}@T{ 23666 - 23667 T}@T{ 23668 - 23669 T}@T{ 23670 Yes 23671 T}@T{ 23672 No 23673 T}@T{ 23674 R 23675 T}@T{ 23676 - 23677 T} 23678 T{ 23679 put.io 23680 T}@T{ 23681 CRC-32 23682 T}@T{ 23683 R/W 23684 T}@T{ 23685 No 23686 T}@T{ 23687 Yes 23688 T}@T{ 23689 R 23690 T}@T{ 23691 - 23692 T} 23693 T{ 23694 Proton Drive 23695 T}@T{ 23696 SHA1 23697 T}@T{ 23698 R/W 23699 T}@T{ 23700 No 23701 T}@T{ 23702 No 23703 T}@T{ 23704 R 23705 T}@T{ 23706 - 23707 T} 23708 T{ 23709 QingStor 23710 T}@T{ 23711 MD5 23712 T}@T{ 23713 - \[u2079] 23714 T}@T{ 23715 No 23716 T}@T{ 23717 No 23718 T}@T{ 23719 R/W 23720 T}@T{ 23721 - 23722 T} 23723 T{ 23724 Quatrix by Maytech 23725 T}@T{ 23726 - 23727 T}@T{ 23728 R/W 23729 T}@T{ 23730 No 23731 T}@T{ 23732 No 23733 T}@T{ 23734 - 23735 T}@T{ 23736 - 23737 T} 23738 T{ 23739 Seafile 23740 T}@T{ 23741 - 23742 T}@T{ 23743 - 23744 T}@T{ 23745 No 23746 T}@T{ 23747 No 23748 T}@T{ 23749 - 23750 T}@T{ 23751 - 23752 T} 23753 T{ 23754 SFTP 23755 T}@T{ 23756 MD5, SHA1 \[S2] 23757 T}@T{ 23758 DR/W 23759 T}@T{ 23760 Depends 23761 T}@T{ 23762 No 23763 T}@T{ 23764 - 23765 T}@T{ 23766 - 23767 T} 23768 T{ 23769 Sia 23770 T}@T{ 23771 - 23772 T}@T{ 23773 - 23774 T}@T{ 23775 No 23776 T}@T{ 23777 No 23778 T}@T{ 23779 - 23780 T}@T{ 23781 - 23782 T} 23783 T{ 23784 SMB 23785 T}@T{ 23786 - 23787 T}@T{ 23788 R/W 23789 T}@T{ 23790 Yes 23791 T}@T{ 23792 No 23793 T}@T{ 23794 - 23795 T}@T{ 23796 - 23797 T} 23798 T{ 23799 SugarSync 23800 T}@T{ 23801 - 23802 T}@T{ 23803 - 23804 T}@T{ 23805 No 23806 T}@T{ 23807 No 23808 T}@T{ 23809 - 23810 T}@T{ 23811 - 23812 T} 23813 T{ 23814 Storj 23815 T}@T{ 23816 - 23817 T}@T{ 23818 R 23819 T}@T{ 23820 No 23821 T}@T{ 23822 No 23823 T}@T{ 23824 - 23825 T}@T{ 23826 - 23827 T} 23828 T{ 23829 Uptobox 23830 T}@T{ 23831 - 23832 T}@T{ 23833 - 23834 T}@T{ 23835 No 23836 T}@T{ 23837 Yes 23838 T}@T{ 23839 - 23840 T}@T{ 23841 - 23842 T} 23843 T{ 23844 WebDAV 23845 T}@T{ 23846 MD5, SHA1 \[S3] 23847 T}@T{ 23848 R \[u2074] 23849 T}@T{ 23850 Depends 23851 T}@T{ 23852 No 23853 T}@T{ 23854 - 23855 T}@T{ 23856 - 23857 T} 23858 T{ 23859 Yandex Disk 23860 T}@T{ 23861 MD5 23862 T}@T{ 23863 R/W 23864 T}@T{ 23865 No 23866 T}@T{ 23867 No 23868 T}@T{ 23869 R 23870 T}@T{ 23871 - 23872 T} 23873 T{ 23874 Zoho WorkDrive 23875 T}@T{ 23876 - 23877 T}@T{ 23878 - 23879 T}@T{ 23880 No 23881 T}@T{ 23882 No 23883 T}@T{ 23884 - 23885 T}@T{ 23886 - 23887 T} 23888 T{ 23889 The local filesystem 23890 T}@T{ 23891 All 23892 T}@T{ 23893 DR/W 23894 T}@T{ 23895 Depends 23896 T}@T{ 23897 No 23898 T}@T{ 23899 - 23900 T}@T{ 23901 DRWU 23902 T} 23903 .TE 23904 .PP 23905 \[S1] Dropbox supports its own custom 23906 hash (https://www.dropbox.com/developers/reference/content-hash). 23907 This is an SHA256 sum of all the 4 MiB block SHA256s. 23908 .PP 23909 \[S2] SFTP supports checksums if the same login has shell access and 23910 \f[C]md5sum\f[R] or \f[C]sha1sum\f[R] as well as \f[C]echo\f[R] are in 23911 the remote\[aq]s PATH. 23912 .PP 23913 \[S3] WebDAV supports hashes when used with Fastmail Files, Owncloud and 23914 Nextcloud only. 23915 .PP 23916 \[u2074] WebDAV supports modtimes when used with Fastmail Files, 23917 Owncloud and Nextcloud only. 23918 .PP 23919 \[u2075] 23920 QuickXorHash (https://docs.microsoft.com/en-us/onedrive/developer/code-snippets/quickxorhash) 23921 is Microsoft\[aq]s own hash. 23922 .PP 23923 \[u2076] Mail.ru uses its own modified SHA1 hash 23924 .PP 23925 \[u2077] pCloud only supports SHA1 (not MD5) in its EU region 23926 .PP 23927 \[u2078] Opendrive does not support creation of duplicate files using 23928 their web client interface or other stock clients, but the underlying 23929 storage platform has been determined to allow duplicate files, and it is 23930 possible to create them with \f[C]rclone\f[R]. 23931 It may be that this is a mistake or an unsupported feature. 23932 .PP 23933 \[u2079] QingStor does not support SetModTime for objects bigger than 5 23934 GiB. 23935 .PP 23936 \[S1]\[u2070] FTP supports modtimes for the major FTP servers, and also 23937 others if they advertised required protocol extensions. 23938 See this (https://rclone.org/ftp/#modification-times) for more details. 23939 .PP 23940 \[S1]\[S1] Internet Archive requires option \f[C]wait_archive\f[R] to be 23941 set to a non-zero value for full modtime support. 23942 .PP 23943 \[S1]\[S2] HiDrive supports its own custom 23944 hash (https://static.hidrive.com/dev/0001). 23945 It combines SHA1 sums for each 4 KiB block hierarchically to a single 23946 top-level sum. 23947 .SS Hash 23948 .PP 23949 The cloud storage system supports various hash types of the objects. 23950 The hashes are used when transferring data as an integrity check and can 23951 be specifically used with the \f[C]--checksum\f[R] flag in syncs and in 23952 the \f[C]check\f[R] command. 23953 .PP 23954 To use the verify checksums when transferring between cloud storage 23955 systems they must support a common hash type. 23956 .SS ModTime 23957 .PP 23958 Almost all cloud storage systems store some sort of timestamp on 23959 objects, but several of them not something that is appropriate to use 23960 for syncing. 23961 E.g. 23962 some backends will only write a timestamp that represents the time of 23963 the upload. 23964 To be relevant for syncing it should be able to store the modification 23965 time of the source object. 23966 If this is not the case, rclone will only check the file size by 23967 default, though can be configured to check the file hash (with the 23968 \f[C]--checksum\f[R] flag). 23969 Ideally it should also be possible to change the timestamp of an 23970 existing file without having to re-upload it. 23971 .PP 23972 .TS 23973 tab(@); 23974 lw(19.4n) lw(50.6n). 23975 T{ 23976 Key 23977 T}@T{ 23978 Explanation 23979 T} 23980 _ 23981 T{ 23982 \f[C]-\f[R] 23983 T}@T{ 23984 ModTimes not supported - times likely the upload time 23985 T} 23986 T{ 23987 \f[C]R\f[R] 23988 T}@T{ 23989 ModTimes supported on files but can\[aq]t be changed without re-upload 23990 T} 23991 T{ 23992 \f[C]R/W\f[R] 23993 T}@T{ 23994 Read and Write ModTimes fully supported on files 23995 T} 23996 T{ 23997 \f[C]DR\f[R] 23998 T}@T{ 23999 ModTimes supported on files and directories but can\[aq]t be changed 24000 without re-upload 24001 T} 24002 T{ 24003 \f[C]DR/W\f[R] 24004 T}@T{ 24005 Read and Write ModTimes fully supported on files and directories 24006 T} 24007 .TE 24008 .PP 24009 Storage systems with a \f[C]-\f[R] in the ModTime column, means the 24010 modification read on objects is not the modification time of the file 24011 when uploaded. 24012 It is most likely the time the file was uploaded, or possibly something 24013 else (like the time the picture was taken in Google Photos). 24014 .PP 24015 Storage systems with a \f[C]R\f[R] (for read-only) in the ModTime 24016 column, means the it keeps modification times on objects, and updates 24017 them when uploading objects, but it does not support changing only the 24018 modification time (\f[C]SetModTime\f[R] operation) without re-uploading, 24019 possibly not even without deleting existing first. 24020 Some operations in rclone, such as \f[C]copy\f[R] and \f[C]sync\f[R] 24021 commands, will automatically check for \f[C]SetModTime\f[R] support and 24022 re-upload if necessary to keep the modification times in sync. 24023 Other commands will not work without \f[C]SetModTime\f[R] support, e.g. 24024 \f[C]touch\f[R] command on an existing file will fail, and changes to 24025 modification time only on a files in a \f[C]mount\f[R] will be silently 24026 ignored. 24027 .PP 24028 Storage systems with \f[C]R/W\f[R] (for read/write) in the ModTime 24029 column, means they do also support modtime-only operations. 24030 .PP 24031 Storage systems with \f[C]D\f[R] in the ModTime column means that the 24032 following symbols apply to directories as well as files. 24033 .SS Case Insensitive 24034 .PP 24035 If a cloud storage systems is case sensitive then it is possible to have 24036 two files which differ only in case, e.g. 24037 \f[C]file.txt\f[R] and \f[C]FILE.txt\f[R]. 24038 If a cloud storage system is case insensitive then that isn\[aq]t 24039 possible. 24040 .PP 24041 This can cause problems when syncing between a case insensitive system 24042 and a case sensitive system. 24043 The symptom of this is that no matter how many times you run the sync it 24044 never completes fully. 24045 .PP 24046 The local filesystem and SFTP may or may not be case sensitive depending 24047 on OS. 24048 .IP \[bu] 2 24049 Windows - usually case insensitive, though case is preserved 24050 .IP \[bu] 2 24051 OSX - usually case insensitive, though it is possible to format case 24052 sensitive 24053 .IP \[bu] 2 24054 Linux - usually case sensitive, but there are case insensitive file 24055 systems (e.g. 24056 FAT formatted USB keys) 24057 .PP 24058 Most of the time this doesn\[aq]t cause any problems as people tend to 24059 avoid files whose name differs only by case even on case sensitive 24060 systems. 24061 .SS Duplicate files 24062 .PP 24063 If a cloud storage system allows duplicate files then it can have two 24064 objects with the same name. 24065 .PP 24066 This confuses rclone greatly when syncing - use the 24067 \f[C]rclone dedupe\f[R] command to rename or remove duplicates. 24068 .SS Restricted filenames 24069 .PP 24070 Some cloud storage systems might have restrictions on the characters 24071 that are usable in file or directory names. 24072 When \f[C]rclone\f[R] detects such a name during a file upload, it will 24073 transparently replace the restricted characters with similar looking 24074 Unicode characters. 24075 To handle the different sets of restricted characters for different 24076 backends, rclone uses something it calls encoding. 24077 .PP 24078 This process is designed to avoid ambiguous file names as much as 24079 possible and allow to move files between many cloud storage systems 24080 transparently. 24081 .PP 24082 The name shown by \f[C]rclone\f[R] to the user or during log output will 24083 only contain a minimal set of replaced characters to ensure correct 24084 formatting and not necessarily the actual name used on the cloud 24085 storage. 24086 .PP 24087 This transformation is reversed when downloading a file or parsing 24088 \f[C]rclone\f[R] arguments. 24089 For example, when uploading a file named \f[C]my file?.txt\f[R] to 24090 Onedrive, it will be displayed as \f[C]my file?.txt\f[R] on the console, 24091 but stored as \f[C]my file\[uFF1F].txt\f[R] to Onedrive (the \f[C]?\f[R] 24092 gets replaced by the similar looking \f[C]\[uFF1F]\f[R] character, the 24093 so-called \[dq]fullwidth question mark\[dq]). 24094 The reverse transformation allows to read a file 24095 \f[C]unusual/name.txt\f[R] from Google Drive, by passing the name 24096 \f[C]unusual\[uFF0F]name.txt\f[R] on the command line (the \f[C]/\f[R] 24097 needs to be replaced by the similar looking \f[C]\[uFF0F]\f[R] 24098 character). 24099 .SS Caveats 24100 .PP 24101 The filename encoding system works well in most cases, at least where 24102 file names are written in English or similar languages. 24103 You might not even notice it: It just works. 24104 In some cases it may lead to issues, though. 24105 E.g. 24106 when file names are written in Chinese, or Japanese, where it is always 24107 the Unicode fullwidth variants of the punctuation marks that are used. 24108 .PP 24109 On Windows, the characters \f[C]:\f[R], \f[C]*\f[R] and \f[C]?\f[R] are 24110 examples of restricted characters. 24111 If these are used in filenames on a remote that supports it, Rclone will 24112 transparently convert them to their fullwidth Unicode variants 24113 \f[C]\[uFF0A]\f[R], \f[C]\[uFF1F]\f[R] and \f[C]\[uFF1A]\f[R] when 24114 downloading to Windows, and back again when uploading. 24115 This way files with names that are not allowed on Windows can still be 24116 stored. 24117 .PP 24118 However, if you have files on your Windows system originally with these 24119 same Unicode characters in their names, they will be included in the 24120 same conversion process. 24121 E.g. 24122 if you create a file in your Windows filesystem with name 24123 \f[C]Test\[uFF1A]1.jpg\f[R], where \f[C]\[uFF1A]\f[R] is the Unicode 24124 fullwidth colon symbol, and use rclone to upload it to Google Drive, 24125 which supports regular \f[C]:\f[R] (halfwidth question mark), rclone 24126 will replace the fullwidth \f[C]:\f[R] with the halfwidth \f[C]:\f[R] 24127 and store the file as \f[C]Test:1.jpg\f[R] in Google Drive. 24128 Since both Windows and Google Drive allows the name 24129 \f[C]Test\[uFF1A]1.jpg\f[R], it would probably be better if rclone just 24130 kept the name as is in this case. 24131 .PP 24132 With the opposite situation; if you have a file named 24133 \f[C]Test:1.jpg\f[R], in your Google Drive, e.g. 24134 uploaded from a Linux system where \f[C]:\f[R] is valid in file names. 24135 Then later use rclone to copy this file to your Windows computer you 24136 will notice that on your local disk it gets renamed to 24137 \f[C]Test\[uFF1A]1.jpg\f[R]. 24138 The original filename is not legal on Windows, due to the \f[C]:\f[R], 24139 and rclone therefore renames it to make the copy possible. 24140 That is all good. 24141 However, this can also lead to an issue: If you already had a 24142 \f[I]different\f[R] file named \f[C]Test\[uFF1A]1.jpg\f[R] on Windows, 24143 and then use rclone to copy either way. 24144 Rclone will then treat the file originally named \f[C]Test:1.jpg\f[R] on 24145 Google Drive and the file originally named \f[C]Test\[uFF1A]1.jpg\f[R] 24146 on Windows as the same file, and replace the contents from one with the 24147 other. 24148 .PP 24149 Its virtually impossible to handle all cases like these correctly in all 24150 situations, but by customizing the encoding option, changing the set of 24151 characters that rclone should convert, you should be able to create a 24152 configuration that works well for your specific situation. 24153 See also the 24154 example (https://rclone.org/overview/#encoding-example-windows) below. 24155 .PP 24156 (Windows was used as an example of a file system with many restricted 24157 characters, and Google drive a storage system with few.) 24158 .SS Default restricted characters 24159 .PP 24160 The table below shows the characters that are replaced by default. 24161 .PP 24162 When a replacement character is found in a filename, this character will 24163 be escaped with the \f[C]\[u201B]\f[R] character to avoid ambiguous file 24164 names. 24165 (e.g. 24166 a file named \f[C]\[u2400].txt\f[R] would shown as 24167 \f[C]\[u201B]\[u2400].txt\f[R]) 24168 .PP 24169 Each cloud storage backend can use a different set of characters, which 24170 will be specified in the documentation for each backend. 24171 .PP 24172 .TS 24173 tab(@); 24174 l c c. 24175 T{ 24176 Character 24177 T}@T{ 24178 Value 24179 T}@T{ 24180 Replacement 24181 T} 24182 _ 24183 T{ 24184 NUL 24185 T}@T{ 24186 0x00 24187 T}@T{ 24188 \[u2400] 24189 T} 24190 T{ 24191 SOH 24192 T}@T{ 24193 0x01 24194 T}@T{ 24195 \[u2401] 24196 T} 24197 T{ 24198 STX 24199 T}@T{ 24200 0x02 24201 T}@T{ 24202 \[u2402] 24203 T} 24204 T{ 24205 ETX 24206 T}@T{ 24207 0x03 24208 T}@T{ 24209 \[u2403] 24210 T} 24211 T{ 24212 EOT 24213 T}@T{ 24214 0x04 24215 T}@T{ 24216 \[u2404] 24217 T} 24218 T{ 24219 ENQ 24220 T}@T{ 24221 0x05 24222 T}@T{ 24223 \[u2405] 24224 T} 24225 T{ 24226 ACK 24227 T}@T{ 24228 0x06 24229 T}@T{ 24230 \[u2406] 24231 T} 24232 T{ 24233 BEL 24234 T}@T{ 24235 0x07 24236 T}@T{ 24237 \[u2407] 24238 T} 24239 T{ 24240 BS 24241 T}@T{ 24242 0x08 24243 T}@T{ 24244 \[u2408] 24245 T} 24246 T{ 24247 HT 24248 T}@T{ 24249 0x09 24250 T}@T{ 24251 \[u2409] 24252 T} 24253 T{ 24254 LF 24255 T}@T{ 24256 0x0A 24257 T}@T{ 24258 \[u240A] 24259 T} 24260 T{ 24261 VT 24262 T}@T{ 24263 0x0B 24264 T}@T{ 24265 \[u240B] 24266 T} 24267 T{ 24268 FF 24269 T}@T{ 24270 0x0C 24271 T}@T{ 24272 \[u240C] 24273 T} 24274 T{ 24275 CR 24276 T}@T{ 24277 0x0D 24278 T}@T{ 24279 \[u240D] 24280 T} 24281 T{ 24282 SO 24283 T}@T{ 24284 0x0E 24285 T}@T{ 24286 \[u240E] 24287 T} 24288 T{ 24289 SI 24290 T}@T{ 24291 0x0F 24292 T}@T{ 24293 \[u240F] 24294 T} 24295 T{ 24296 DLE 24297 T}@T{ 24298 0x10 24299 T}@T{ 24300 \[u2410] 24301 T} 24302 T{ 24303 DC1 24304 T}@T{ 24305 0x11 24306 T}@T{ 24307 \[u2411] 24308 T} 24309 T{ 24310 DC2 24311 T}@T{ 24312 0x12 24313 T}@T{ 24314 \[u2412] 24315 T} 24316 T{ 24317 DC3 24318 T}@T{ 24319 0x13 24320 T}@T{ 24321 \[u2413] 24322 T} 24323 T{ 24324 DC4 24325 T}@T{ 24326 0x14 24327 T}@T{ 24328 \[u2414] 24329 T} 24330 T{ 24331 NAK 24332 T}@T{ 24333 0x15 24334 T}@T{ 24335 \[u2415] 24336 T} 24337 T{ 24338 SYN 24339 T}@T{ 24340 0x16 24341 T}@T{ 24342 \[u2416] 24343 T} 24344 T{ 24345 ETB 24346 T}@T{ 24347 0x17 24348 T}@T{ 24349 \[u2417] 24350 T} 24351 T{ 24352 CAN 24353 T}@T{ 24354 0x18 24355 T}@T{ 24356 \[u2418] 24357 T} 24358 T{ 24359 EM 24360 T}@T{ 24361 0x19 24362 T}@T{ 24363 \[u2419] 24364 T} 24365 T{ 24366 SUB 24367 T}@T{ 24368 0x1A 24369 T}@T{ 24370 \[u241A] 24371 T} 24372 T{ 24373 ESC 24374 T}@T{ 24375 0x1B 24376 T}@T{ 24377 \[u241B] 24378 T} 24379 T{ 24380 FS 24381 T}@T{ 24382 0x1C 24383 T}@T{ 24384 \[u241C] 24385 T} 24386 T{ 24387 GS 24388 T}@T{ 24389 0x1D 24390 T}@T{ 24391 \[u241D] 24392 T} 24393 T{ 24394 RS 24395 T}@T{ 24396 0x1E 24397 T}@T{ 24398 \[u241E] 24399 T} 24400 T{ 24401 US 24402 T}@T{ 24403 0x1F 24404 T}@T{ 24405 \[u241F] 24406 T} 24407 T{ 24408 / 24409 T}@T{ 24410 0x2F 24411 T}@T{ 24412 \[uFF0F] 24413 T} 24414 T{ 24415 DEL 24416 T}@T{ 24417 0x7F 24418 T}@T{ 24419 \[u2421] 24420 T} 24421 .TE 24422 .PP 24423 The default encoding will also encode these file names as they are 24424 problematic with many cloud storage systems. 24425 .PP 24426 .TS 24427 tab(@); 24428 l c. 24429 T{ 24430 File name 24431 T}@T{ 24432 Replacement 24433 T} 24434 _ 24435 T{ 24436 \&. 24437 T}@T{ 24438 \[uFF0E] 24439 T} 24440 T{ 24441 \&.. 24442 T}@T{ 24443 \[uFF0E]\[uFF0E] 24444 T} 24445 .TE 24446 .SS Invalid UTF-8 bytes 24447 .PP 24448 Some backends only support a sequence of well formed UTF-8 bytes as file 24449 or directory names. 24450 .PP 24451 In this case all invalid UTF-8 bytes will be replaced with a quoted 24452 representation of the byte value to allow uploading a file to such a 24453 backend. 24454 For example, the invalid byte \f[C]0xFE\f[R] will be encoded as 24455 \f[C]\[u201B]FE\f[R]. 24456 .PP 24457 A common source of invalid UTF-8 bytes are local filesystems, that store 24458 names in a different encoding than UTF-8 or UTF-16, like latin1. 24459 See the local filenames (https://rclone.org/local/#filenames) section 24460 for details. 24461 .SS Encoding option 24462 .PP 24463 Most backends have an encoding option, specified as a flag 24464 \f[C]--backend-encoding\f[R] where \f[C]backend\f[R] is the name of the 24465 backend, or as a config parameter \f[C]encoding\f[R] (you\[aq]ll need to 24466 select the Advanced config in \f[C]rclone config\f[R] to see it). 24467 .PP 24468 This will have default value which encodes and decodes characters in 24469 such a way as to preserve the maximum number of characters (see above). 24470 .PP 24471 However this can be incorrect in some scenarios, for example if you have 24472 a Windows file system with Unicode fullwidth characters 24473 \f[C]\[uFF0A]\f[R], \f[C]\[uFF1F]\f[R] or \f[C]\[uFF1A]\f[R], that you 24474 want to remain as those characters on the remote rather than being 24475 translated to regular (halfwidth) \f[C]*\f[R], \f[C]?\f[R] and 24476 \f[C]:\f[R]. 24477 .PP 24478 The \f[C]--backend-encoding\f[R] flags allow you to change that. 24479 You can disable the encoding completely with 24480 \f[C]--backend-encoding None\f[R] or set \f[C]encoding = None\f[R] in 24481 the config file. 24482 .PP 24483 Encoding takes a comma separated list of encodings. 24484 You can see the list of all possible values by passing an invalid value 24485 to this flag, e.g. 24486 \f[C]--local-encoding \[dq]help\[dq]\f[R]. 24487 The command \f[C]rclone help flags encoding\f[R] will show you the 24488 defaults for the backends. 24489 .PP 24490 .TS 24491 tab(@); 24492 lw(21.7n) lw(24.1n) lw(24.1n). 24493 T{ 24494 Encoding 24495 T}@T{ 24496 Characters 24497 T}@T{ 24498 Encoded as 24499 T} 24500 _ 24501 T{ 24502 Asterisk 24503 T}@T{ 24504 \f[C]*\f[R] 24505 T}@T{ 24506 \f[C]\[uFF0A]\f[R] 24507 T} 24508 T{ 24509 BackQuote 24510 T}@T{ 24511 \f[C]\[ga]\f[R] 24512 T}@T{ 24513 \f[C]\[uFF40]\f[R] 24514 T} 24515 T{ 24516 BackSlash 24517 T}@T{ 24518 \f[C]\[rs]\f[R] 24519 T}@T{ 24520 \f[C]\[uFF3C]\f[R] 24521 T} 24522 T{ 24523 Colon 24524 T}@T{ 24525 \f[C]:\f[R] 24526 T}@T{ 24527 \f[C]\[uFF1A]\f[R] 24528 T} 24529 T{ 24530 CrLf 24531 T}@T{ 24532 CR 0x0D, LF 0x0A 24533 T}@T{ 24534 \f[C]\[u240D]\f[R], \f[C]\[u240A]\f[R] 24535 T} 24536 T{ 24537 Ctl 24538 T}@T{ 24539 All control characters 0x00-0x1F 24540 T}@T{ 24541 \f[C]\[u2400]\[u2401]\[u2402]\[u2403]\[u2404]\[u2405]\[u2406]\[u2407]\[u2408]\[u2409]\[u240A]\[u240B]\[u240C]\[u240D]\[u240E]\[u240F]\[u2410]\[u2411]\[u2412]\[u2413]\[u2414]\[u2415]\[u2416]\[u2417]\[u2418]\[u2419]\[u241A]\[u241B]\[u241C]\[u241D]\[u241E]\[u241F]\f[R] 24542 T} 24543 T{ 24544 Del 24545 T}@T{ 24546 DEL 0x7F 24547 T}@T{ 24548 \f[C]\[u2421]\f[R] 24549 T} 24550 T{ 24551 Dollar 24552 T}@T{ 24553 \f[C]$\f[R] 24554 T}@T{ 24555 \f[C]\[uFF04]\f[R] 24556 T} 24557 T{ 24558 Dot 24559 T}@T{ 24560 \f[C].\f[R] or \f[C]..\f[R] as entire string 24561 T}@T{ 24562 \f[C]\[uFF0E]\f[R], \f[C]\[uFF0E]\[uFF0E]\f[R] 24563 T} 24564 T{ 24565 DoubleQuote 24566 T}@T{ 24567 \f[C]\[dq]\f[R] 24568 T}@T{ 24569 \f[C]\[uFF02]\f[R] 24570 T} 24571 T{ 24572 Hash 24573 T}@T{ 24574 \f[C]#\f[R] 24575 T}@T{ 24576 \f[C]\[uFF03]\f[R] 24577 T} 24578 T{ 24579 InvalidUtf8 24580 T}@T{ 24581 An invalid UTF-8 character (e.g. 24582 latin1) 24583 T}@T{ 24584 \f[C]\[uFFFD]\f[R] 24585 T} 24586 T{ 24587 LeftCrLfHtVt 24588 T}@T{ 24589 CR 0x0D, LF 0x0A, HT 0x09, VT 0x0B on the left of a string 24590 T}@T{ 24591 \f[C]\[u240D]\f[R], \f[C]\[u240A]\f[R], \f[C]\[u2409]\f[R], 24592 \f[C]\[u240B]\f[R] 24593 T} 24594 T{ 24595 LeftPeriod 24596 T}@T{ 24597 \f[C].\f[R] on the left of a string 24598 T}@T{ 24599 \f[C].\f[R] 24600 T} 24601 T{ 24602 LeftSpace 24603 T}@T{ 24604 SPACE on the left of a string 24605 T}@T{ 24606 \f[C]\[u2420]\f[R] 24607 T} 24608 T{ 24609 LeftTilde 24610 T}@T{ 24611 \f[C]\[ti]\f[R] on the left of a string 24612 T}@T{ 24613 \f[C]\[uFF5E]\f[R] 24614 T} 24615 T{ 24616 LtGt 24617 T}@T{ 24618 \f[C]<\f[R], \f[C]>\f[R] 24619 T}@T{ 24620 \f[C]\[uFF1C]\f[R], \f[C]\[uFF1E]\f[R] 24621 T} 24622 T{ 24623 None 24624 T}@T{ 24625 No characters are encoded 24626 T}@T{ 24627 T} 24628 T{ 24629 Percent 24630 T}@T{ 24631 \f[C]%\f[R] 24632 T}@T{ 24633 \f[C]\[uFF05]\f[R] 24634 T} 24635 T{ 24636 Pipe 24637 T}@T{ 24638 | 24639 T}@T{ 24640 \f[C]\[uFF5C]\f[R] 24641 T} 24642 T{ 24643 Question 24644 T}@T{ 24645 \f[C]?\f[R] 24646 T}@T{ 24647 \f[C]\[uFF1F]\f[R] 24648 T} 24649 T{ 24650 RightCrLfHtVt 24651 T}@T{ 24652 CR 0x0D, LF 0x0A, HT 0x09, VT 0x0B on the right of a string 24653 T}@T{ 24654 \f[C]\[u240D]\f[R], \f[C]\[u240A]\f[R], \f[C]\[u2409]\f[R], 24655 \f[C]\[u240B]\f[R] 24656 T} 24657 T{ 24658 RightPeriod 24659 T}@T{ 24660 \f[C].\f[R] on the right of a string 24661 T}@T{ 24662 \f[C].\f[R] 24663 T} 24664 T{ 24665 RightSpace 24666 T}@T{ 24667 SPACE on the right of a string 24668 T}@T{ 24669 \f[C]\[u2420]\f[R] 24670 T} 24671 T{ 24672 Semicolon 24673 T}@T{ 24674 \f[C];\f[R] 24675 T}@T{ 24676 \f[C]\[uFF1B]\f[R] 24677 T} 24678 T{ 24679 SingleQuote 24680 T}@T{ 24681 \f[C]\[aq]\f[R] 24682 T}@T{ 24683 \f[C]\[uFF07]\f[R] 24684 T} 24685 T{ 24686 Slash 24687 T}@T{ 24688 \f[C]/\f[R] 24689 T}@T{ 24690 \f[C]\[uFF0F]\f[R] 24691 T} 24692 T{ 24693 SquareBracket 24694 T}@T{ 24695 \f[C][\f[R], \f[C]]\f[R] 24696 T}@T{ 24697 \f[C]\[uFF3B]\f[R], \f[C]\[uFF3D]\f[R] 24698 T} 24699 .TE 24700 .SS Encoding example: FTP 24701 .PP 24702 To take a specific example, the FTP backend\[aq]s default encoding is 24703 .IP 24704 .nf 24705 \f[C] 24706 --ftp-encoding \[dq]Slash,Del,Ctl,RightSpace,Dot\[dq] 24707 \f[R] 24708 .fi 24709 .PP 24710 However, let\[aq]s say the FTP server is running on Windows and 24711 can\[aq]t have any of the invalid Windows characters in file names. 24712 You are backing up Linux servers to this FTP server which do have those 24713 characters in file names. 24714 So you would add the Windows set which are 24715 .IP 24716 .nf 24717 \f[C] 24718 Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Ctl,RightSpace,RightPeriod,InvalidUtf8,Dot 24719 \f[R] 24720 .fi 24721 .PP 24722 to the existing ones, giving: 24723 .IP 24724 .nf 24725 \f[C] 24726 Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Ctl,RightSpace,RightPeriod,InvalidUtf8,Dot,Del,RightSpace 24727 \f[R] 24728 .fi 24729 .PP 24730 This can be specified using the \f[C]--ftp-encoding\f[R] flag or using 24731 an \f[C]encoding\f[R] parameter in the config file. 24732 .SS Encoding example: Windows 24733 .PP 24734 As a nother example, take a Windows system where there is a file with 24735 name \f[C]Test\[uFF1A]1.jpg\f[R], where \f[C]\[uFF1A]\f[R] is the 24736 Unicode fullwidth colon symbol. 24737 When using rclone to copy this to a remote which supports \f[C]:\f[R], 24738 the regular (halfwidth) colon (such as Google Drive), you will notice 24739 that the file gets renamed to \f[C]Test:1.jpg\f[R]. 24740 .PP 24741 To avoid this you can change the set of characters rclone should convert 24742 for the local filesystem, using command-line argument 24743 \f[C]--local-encoding\f[R]. 24744 Rclone\[aq]s default behavior on Windows corresponds to 24745 .IP 24746 .nf 24747 \f[C] 24748 --local-encoding \[dq]Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Ctl,RightSpace,RightPeriod,InvalidUtf8,Dot\[dq] 24749 \f[R] 24750 .fi 24751 .PP 24752 If you want to use fullwidth characters \f[C]\[uFF1A]\f[R], 24753 \f[C]\[uFF0A]\f[R] and \f[C]\[uFF1F]\f[R] in your filenames without 24754 rclone changing them when uploading to a remote, then set the same as 24755 the default value but without \f[C]Colon,Question,Asterisk\f[R]: 24756 .IP 24757 .nf 24758 \f[C] 24759 --local-encoding \[dq]Slash,LtGt,DoubleQuote,Pipe,BackSlash,Ctl,RightSpace,RightPeriod,InvalidUtf8,Dot\[dq] 24760 \f[R] 24761 .fi 24762 .PP 24763 Alternatively, you can disable the conversion of any characters with 24764 \f[C]--local-encoding None\f[R]. 24765 .PP 24766 Instead of using command-line argument \f[C]--local-encoding\f[R], you 24767 may also set it as environment 24768 variable (https://rclone.org/docs/#environment-variables) 24769 \f[C]RCLONE_LOCAL_ENCODING\f[R], or 24770 configure (https://rclone.org/docs/#configure) a remote of type 24771 \f[C]local\f[R] in your config, and set the \f[C]encoding\f[R] option 24772 there. 24773 .PP 24774 The risk by doing this is that if you have a filename with the regular 24775 (halfwidth) \f[C]:\f[R], \f[C]*\f[R] and \f[C]?\f[R] in your cloud 24776 storage, and you try to download it to your Windows filesystem, this 24777 will fail. 24778 These characters are not valid in filenames on Windows, and you have 24779 told rclone not to work around this by converting them to valid 24780 fullwidth variants. 24781 .SS MIME Type 24782 .PP 24783 MIME types (also known as media types) classify types of documents using 24784 a simple text classification, e.g. 24785 \f[C]text/html\f[R] or \f[C]application/pdf\f[R]. 24786 .PP 24787 Some cloud storage systems support reading (\f[C]R\f[R]) the MIME type 24788 of objects and some support writing (\f[C]W\f[R]) the MIME type of 24789 objects. 24790 .PP 24791 The MIME type can be important if you are serving files directly to HTTP 24792 from the storage system. 24793 .PP 24794 If you are copying from a remote which supports reading (\f[C]R\f[R]) to 24795 a remote which supports writing (\f[C]W\f[R]) then rclone will preserve 24796 the MIME types. 24797 Otherwise they will be guessed from the extension, or the remote itself 24798 may assign the MIME type. 24799 .SS Metadata 24800 .PP 24801 Backends may or may support reading or writing metadata. 24802 They may support reading and writing system metadata (metadata intrinsic 24803 to that backend) and/or user metadata (general purpose metadata). 24804 .PP 24805 The levels of metadata support are 24806 .PP 24807 .TS 24808 tab(@); 24809 lw(19.4n) lw(50.6n). 24810 T{ 24811 Key 24812 T}@T{ 24813 Explanation 24814 T} 24815 _ 24816 T{ 24817 \f[C]R\f[R] 24818 T}@T{ 24819 Read only System Metadata on files only 24820 T} 24821 T{ 24822 \f[C]RW\f[R] 24823 T}@T{ 24824 Read and write System Metadata on files only 24825 T} 24826 T{ 24827 \f[C]RWU\f[R] 24828 T}@T{ 24829 Read and write System Metadata and read and write User Metadata on files 24830 only 24831 T} 24832 T{ 24833 \f[C]DR\f[R] 24834 T}@T{ 24835 Read only System Metadata on files and directories 24836 T} 24837 T{ 24838 \f[C]DRW\f[R] 24839 T}@T{ 24840 Read and write System Metadata on files and directories 24841 T} 24842 T{ 24843 \f[C]DRWU\f[R] 24844 T}@T{ 24845 Read and write System Metadata and read and write User Metadata on files 24846 and directories 24847 T} 24848 .TE 24849 .PP 24850 See the metadata docs (https://rclone.org/docs/#metadata) for more info. 24851 .SS Optional Features 24852 .PP 24853 All rclone remotes support a base command set. 24854 Other features depend upon backend-specific capabilities. 24855 .PP 24856 .TS 24857 tab(@); 24858 lw(14.4n) cw(3.6n) cw(3.1n) cw(3.1n) cw(4.6n) cw(4.6n) cw(3.6n) cw(7.2n) lw(9.8n) cw(7.2n) cw(3.6n) cw(5.1n). 24859 T{ 24860 Name 24861 T}@T{ 24862 Purge 24863 T}@T{ 24864 Copy 24865 T}@T{ 24866 Move 24867 T}@T{ 24868 DirMove 24869 T}@T{ 24870 CleanUp 24871 T}@T{ 24872 ListR 24873 T}@T{ 24874 StreamUpload 24875 T}@T{ 24876 MultithreadUpload 24877 T}@T{ 24878 LinkSharing 24879 T}@T{ 24880 About 24881 T}@T{ 24882 EmptyDir 24883 T} 24884 _ 24885 T{ 24886 1Fichier 24887 T}@T{ 24888 No 24889 T}@T{ 24890 Yes 24891 T}@T{ 24892 Yes 24893 T}@T{ 24894 No 24895 T}@T{ 24896 No 24897 T}@T{ 24898 No 24899 T}@T{ 24900 No 24901 T}@T{ 24902 No 24903 T}@T{ 24904 Yes 24905 T}@T{ 24906 No 24907 T}@T{ 24908 Yes 24909 T} 24910 T{ 24911 Akamai Netstorage 24912 T}@T{ 24913 Yes 24914 T}@T{ 24915 No 24916 T}@T{ 24917 No 24918 T}@T{ 24919 No 24920 T}@T{ 24921 No 24922 T}@T{ 24923 Yes 24924 T}@T{ 24925 Yes 24926 T}@T{ 24927 No 24928 T}@T{ 24929 No 24930 T}@T{ 24931 No 24932 T}@T{ 24933 Yes 24934 T} 24935 T{ 24936 Amazon S3 (or S3 compatible) 24937 T}@T{ 24938 No 24939 T}@T{ 24940 Yes 24941 T}@T{ 24942 No 24943 T}@T{ 24944 No 24945 T}@T{ 24946 Yes 24947 T}@T{ 24948 Yes 24949 T}@T{ 24950 Yes 24951 T}@T{ 24952 Yes 24953 T}@T{ 24954 Yes 24955 T}@T{ 24956 No 24957 T}@T{ 24958 No 24959 T} 24960 T{ 24961 Backblaze B2 24962 T}@T{ 24963 No 24964 T}@T{ 24965 Yes 24966 T}@T{ 24967 No 24968 T}@T{ 24969 No 24970 T}@T{ 24971 Yes 24972 T}@T{ 24973 Yes 24974 T}@T{ 24975 Yes 24976 T}@T{ 24977 Yes 24978 T}@T{ 24979 Yes 24980 T}@T{ 24981 No 24982 T}@T{ 24983 No 24984 T} 24985 T{ 24986 Box 24987 T}@T{ 24988 Yes 24989 T}@T{ 24990 Yes 24991 T}@T{ 24992 Yes 24993 T}@T{ 24994 Yes 24995 T}@T{ 24996 Yes 24997 T}@T{ 24998 No 24999 T}@T{ 25000 Yes 25001 T}@T{ 25002 No 25003 T}@T{ 25004 Yes 25005 T}@T{ 25006 Yes 25007 T}@T{ 25008 Yes 25009 T} 25010 T{ 25011 Citrix ShareFile 25012 T}@T{ 25013 Yes 25014 T}@T{ 25015 Yes 25016 T}@T{ 25017 Yes 25018 T}@T{ 25019 Yes 25020 T}@T{ 25021 No 25022 T}@T{ 25023 No 25024 T}@T{ 25025 No 25026 T}@T{ 25027 No 25028 T}@T{ 25029 No 25030 T}@T{ 25031 No 25032 T}@T{ 25033 Yes 25034 T} 25035 T{ 25036 Dropbox 25037 T}@T{ 25038 Yes 25039 T}@T{ 25040 Yes 25041 T}@T{ 25042 Yes 25043 T}@T{ 25044 Yes 25045 T}@T{ 25046 No 25047 T}@T{ 25048 No 25049 T}@T{ 25050 Yes 25051 T}@T{ 25052 No 25053 T}@T{ 25054 Yes 25055 T}@T{ 25056 Yes 25057 T}@T{ 25058 Yes 25059 T} 25060 T{ 25061 Enterprise File Fabric 25062 T}@T{ 25063 Yes 25064 T}@T{ 25065 Yes 25066 T}@T{ 25067 Yes 25068 T}@T{ 25069 Yes 25070 T}@T{ 25071 Yes 25072 T}@T{ 25073 No 25074 T}@T{ 25075 No 25076 T}@T{ 25077 No 25078 T}@T{ 25079 No 25080 T}@T{ 25081 No 25082 T}@T{ 25083 Yes 25084 T} 25085 T{ 25086 FTP 25087 T}@T{ 25088 No 25089 T}@T{ 25090 No 25091 T}@T{ 25092 Yes 25093 T}@T{ 25094 Yes 25095 T}@T{ 25096 No 25097 T}@T{ 25098 No 25099 T}@T{ 25100 Yes 25101 T}@T{ 25102 No 25103 T}@T{ 25104 No 25105 T}@T{ 25106 No 25107 T}@T{ 25108 Yes 25109 T} 25110 T{ 25111 Google Cloud Storage 25112 T}@T{ 25113 Yes 25114 T}@T{ 25115 Yes 25116 T}@T{ 25117 No 25118 T}@T{ 25119 No 25120 T}@T{ 25121 No 25122 T}@T{ 25123 Yes 25124 T}@T{ 25125 Yes 25126 T}@T{ 25127 No 25128 T}@T{ 25129 No 25130 T}@T{ 25131 No 25132 T}@T{ 25133 No 25134 T} 25135 T{ 25136 Google Drive 25137 T}@T{ 25138 Yes 25139 T}@T{ 25140 Yes 25141 T}@T{ 25142 Yes 25143 T}@T{ 25144 Yes 25145 T}@T{ 25146 Yes 25147 T}@T{ 25148 Yes 25149 T}@T{ 25150 Yes 25151 T}@T{ 25152 No 25153 T}@T{ 25154 Yes 25155 T}@T{ 25156 Yes 25157 T}@T{ 25158 Yes 25159 T} 25160 T{ 25161 Google Photos 25162 T}@T{ 25163 No 25164 T}@T{ 25165 No 25166 T}@T{ 25167 No 25168 T}@T{ 25169 No 25170 T}@T{ 25171 No 25172 T}@T{ 25173 No 25174 T}@T{ 25175 No 25176 T}@T{ 25177 No 25178 T}@T{ 25179 No 25180 T}@T{ 25181 No 25182 T}@T{ 25183 No 25184 T} 25185 T{ 25186 HDFS 25187 T}@T{ 25188 Yes 25189 T}@T{ 25190 No 25191 T}@T{ 25192 Yes 25193 T}@T{ 25194 Yes 25195 T}@T{ 25196 No 25197 T}@T{ 25198 No 25199 T}@T{ 25200 Yes 25201 T}@T{ 25202 No 25203 T}@T{ 25204 No 25205 T}@T{ 25206 Yes 25207 T}@T{ 25208 Yes 25209 T} 25210 T{ 25211 HiDrive 25212 T}@T{ 25213 Yes 25214 T}@T{ 25215 Yes 25216 T}@T{ 25217 Yes 25218 T}@T{ 25219 Yes 25220 T}@T{ 25221 No 25222 T}@T{ 25223 No 25224 T}@T{ 25225 Yes 25226 T}@T{ 25227 No 25228 T}@T{ 25229 No 25230 T}@T{ 25231 No 25232 T}@T{ 25233 Yes 25234 T} 25235 T{ 25236 HTTP 25237 T}@T{ 25238 No 25239 T}@T{ 25240 No 25241 T}@T{ 25242 No 25243 T}@T{ 25244 No 25245 T}@T{ 25246 No 25247 T}@T{ 25248 No 25249 T}@T{ 25250 No 25251 T}@T{ 25252 No 25253 T}@T{ 25254 No 25255 T}@T{ 25256 No 25257 T}@T{ 25258 Yes 25259 T} 25260 T{ 25261 ImageKit 25262 T}@T{ 25263 Yes 25264 T}@T{ 25265 Yes 25266 T}@T{ 25267 Yes 25268 T}@T{ 25269 No 25270 T}@T{ 25271 No 25272 T}@T{ 25273 No 25274 T}@T{ 25275 No 25276 T}@T{ 25277 No 25278 T}@T{ 25279 No 25280 T}@T{ 25281 No 25282 T}@T{ 25283 Yes 25284 T} 25285 T{ 25286 Internet Archive 25287 T}@T{ 25288 No 25289 T}@T{ 25290 Yes 25291 T}@T{ 25292 No 25293 T}@T{ 25294 No 25295 T}@T{ 25296 Yes 25297 T}@T{ 25298 Yes 25299 T}@T{ 25300 No 25301 T}@T{ 25302 No 25303 T}@T{ 25304 Yes 25305 T}@T{ 25306 Yes 25307 T}@T{ 25308 No 25309 T} 25310 T{ 25311 Jottacloud 25312 T}@T{ 25313 Yes 25314 T}@T{ 25315 Yes 25316 T}@T{ 25317 Yes 25318 T}@T{ 25319 Yes 25320 T}@T{ 25321 Yes 25322 T}@T{ 25323 Yes 25324 T}@T{ 25325 No 25326 T}@T{ 25327 No 25328 T}@T{ 25329 Yes 25330 T}@T{ 25331 Yes 25332 T}@T{ 25333 Yes 25334 T} 25335 T{ 25336 Koofr 25337 T}@T{ 25338 Yes 25339 T}@T{ 25340 Yes 25341 T}@T{ 25342 Yes 25343 T}@T{ 25344 Yes 25345 T}@T{ 25346 No 25347 T}@T{ 25348 No 25349 T}@T{ 25350 Yes 25351 T}@T{ 25352 No 25353 T}@T{ 25354 Yes 25355 T}@T{ 25356 Yes 25357 T}@T{ 25358 Yes 25359 T} 25360 T{ 25361 Mail.ru Cloud 25362 T}@T{ 25363 Yes 25364 T}@T{ 25365 Yes 25366 T}@T{ 25367 Yes 25368 T}@T{ 25369 Yes 25370 T}@T{ 25371 Yes 25372 T}@T{ 25373 No 25374 T}@T{ 25375 No 25376 T}@T{ 25377 No 25378 T}@T{ 25379 Yes 25380 T}@T{ 25381 Yes 25382 T}@T{ 25383 Yes 25384 T} 25385 T{ 25386 Mega 25387 T}@T{ 25388 Yes 25389 T}@T{ 25390 No 25391 T}@T{ 25392 Yes 25393 T}@T{ 25394 Yes 25395 T}@T{ 25396 Yes 25397 T}@T{ 25398 No 25399 T}@T{ 25400 No 25401 T}@T{ 25402 No 25403 T}@T{ 25404 Yes 25405 T}@T{ 25406 Yes 25407 T}@T{ 25408 Yes 25409 T} 25410 T{ 25411 Memory 25412 T}@T{ 25413 No 25414 T}@T{ 25415 Yes 25416 T}@T{ 25417 No 25418 T}@T{ 25419 No 25420 T}@T{ 25421 No 25422 T}@T{ 25423 Yes 25424 T}@T{ 25425 Yes 25426 T}@T{ 25427 No 25428 T}@T{ 25429 No 25430 T}@T{ 25431 No 25432 T}@T{ 25433 No 25434 T} 25435 T{ 25436 Microsoft Azure Blob Storage 25437 T}@T{ 25438 Yes 25439 T}@T{ 25440 Yes 25441 T}@T{ 25442 No 25443 T}@T{ 25444 No 25445 T}@T{ 25446 No 25447 T}@T{ 25448 Yes 25449 T}@T{ 25450 Yes 25451 T}@T{ 25452 Yes 25453 T}@T{ 25454 No 25455 T}@T{ 25456 No 25457 T}@T{ 25458 No 25459 T} 25460 T{ 25461 Microsoft Azure Files Storage 25462 T}@T{ 25463 No 25464 T}@T{ 25465 Yes 25466 T}@T{ 25467 Yes 25468 T}@T{ 25469 Yes 25470 T}@T{ 25471 No 25472 T}@T{ 25473 No 25474 T}@T{ 25475 Yes 25476 T}@T{ 25477 Yes 25478 T}@T{ 25479 No 25480 T}@T{ 25481 Yes 25482 T}@T{ 25483 Yes 25484 T} 25485 T{ 25486 Microsoft OneDrive 25487 T}@T{ 25488 Yes 25489 T}@T{ 25490 Yes 25491 T}@T{ 25492 Yes 25493 T}@T{ 25494 Yes 25495 T}@T{ 25496 Yes 25497 T}@T{ 25498 Yes \[u2075] 25499 T}@T{ 25500 No 25501 T}@T{ 25502 No 25503 T}@T{ 25504 Yes 25505 T}@T{ 25506 Yes 25507 T}@T{ 25508 Yes 25509 T} 25510 T{ 25511 OpenDrive 25512 T}@T{ 25513 Yes 25514 T}@T{ 25515 Yes 25516 T}@T{ 25517 Yes 25518 T}@T{ 25519 Yes 25520 T}@T{ 25521 No 25522 T}@T{ 25523 No 25524 T}@T{ 25525 No 25526 T}@T{ 25527 No 25528 T}@T{ 25529 No 25530 T}@T{ 25531 No 25532 T}@T{ 25533 Yes 25534 T} 25535 T{ 25536 OpenStack Swift 25537 T}@T{ 25538 Yes \[S1] 25539 T}@T{ 25540 Yes 25541 T}@T{ 25542 No 25543 T}@T{ 25544 No 25545 T}@T{ 25546 No 25547 T}@T{ 25548 Yes 25549 T}@T{ 25550 Yes 25551 T}@T{ 25552 No 25553 T}@T{ 25554 No 25555 T}@T{ 25556 Yes 25557 T}@T{ 25558 No 25559 T} 25560 T{ 25561 Oracle Object Storage 25562 T}@T{ 25563 No 25564 T}@T{ 25565 Yes 25566 T}@T{ 25567 No 25568 T}@T{ 25569 No 25570 T}@T{ 25571 Yes 25572 T}@T{ 25573 Yes 25574 T}@T{ 25575 Yes 25576 T}@T{ 25577 Yes 25578 T}@T{ 25579 No 25580 T}@T{ 25581 No 25582 T}@T{ 25583 No 25584 T} 25585 T{ 25586 pCloud 25587 T}@T{ 25588 Yes 25589 T}@T{ 25590 Yes 25591 T}@T{ 25592 Yes 25593 T}@T{ 25594 Yes 25595 T}@T{ 25596 Yes 25597 T}@T{ 25598 No 25599 T}@T{ 25600 No 25601 T}@T{ 25602 No 25603 T}@T{ 25604 Yes 25605 T}@T{ 25606 Yes 25607 T}@T{ 25608 Yes 25609 T} 25610 T{ 25611 PikPak 25612 T}@T{ 25613 Yes 25614 T}@T{ 25615 Yes 25616 T}@T{ 25617 Yes 25618 T}@T{ 25619 Yes 25620 T}@T{ 25621 Yes 25622 T}@T{ 25623 No 25624 T}@T{ 25625 No 25626 T}@T{ 25627 No 25628 T}@T{ 25629 Yes 25630 T}@T{ 25631 Yes 25632 T}@T{ 25633 Yes 25634 T} 25635 T{ 25636 premiumize.me 25637 T}@T{ 25638 Yes 25639 T}@T{ 25640 No 25641 T}@T{ 25642 Yes 25643 T}@T{ 25644 Yes 25645 T}@T{ 25646 No 25647 T}@T{ 25648 No 25649 T}@T{ 25650 No 25651 T}@T{ 25652 No 25653 T}@T{ 25654 Yes 25655 T}@T{ 25656 Yes 25657 T}@T{ 25658 Yes 25659 T} 25660 T{ 25661 put.io 25662 T}@T{ 25663 Yes 25664 T}@T{ 25665 No 25666 T}@T{ 25667 Yes 25668 T}@T{ 25669 Yes 25670 T}@T{ 25671 Yes 25672 T}@T{ 25673 No 25674 T}@T{ 25675 Yes 25676 T}@T{ 25677 No 25678 T}@T{ 25679 No 25680 T}@T{ 25681 Yes 25682 T}@T{ 25683 Yes 25684 T} 25685 T{ 25686 Proton Drive 25687 T}@T{ 25688 Yes 25689 T}@T{ 25690 No 25691 T}@T{ 25692 Yes 25693 T}@T{ 25694 Yes 25695 T}@T{ 25696 Yes 25697 T}@T{ 25698 No 25699 T}@T{ 25700 No 25701 T}@T{ 25702 No 25703 T}@T{ 25704 No 25705 T}@T{ 25706 Yes 25707 T}@T{ 25708 Yes 25709 T} 25710 T{ 25711 QingStor 25712 T}@T{ 25713 No 25714 T}@T{ 25715 Yes 25716 T}@T{ 25717 No 25718 T}@T{ 25719 No 25720 T}@T{ 25721 Yes 25722 T}@T{ 25723 Yes 25724 T}@T{ 25725 No 25726 T}@T{ 25727 No 25728 T}@T{ 25729 No 25730 T}@T{ 25731 No 25732 T}@T{ 25733 No 25734 T} 25735 T{ 25736 Quatrix by Maytech 25737 T}@T{ 25738 Yes 25739 T}@T{ 25740 Yes 25741 T}@T{ 25742 Yes 25743 T}@T{ 25744 Yes 25745 T}@T{ 25746 No 25747 T}@T{ 25748 No 25749 T}@T{ 25750 No 25751 T}@T{ 25752 No 25753 T}@T{ 25754 No 25755 T}@T{ 25756 Yes 25757 T}@T{ 25758 Yes 25759 T} 25760 T{ 25761 Seafile 25762 T}@T{ 25763 Yes 25764 T}@T{ 25765 Yes 25766 T}@T{ 25767 Yes 25768 T}@T{ 25769 Yes 25770 T}@T{ 25771 Yes 25772 T}@T{ 25773 Yes 25774 T}@T{ 25775 Yes 25776 T}@T{ 25777 No 25778 T}@T{ 25779 Yes 25780 T}@T{ 25781 Yes 25782 T}@T{ 25783 Yes 25784 T} 25785 T{ 25786 SFTP 25787 T}@T{ 25788 No 25789 T}@T{ 25790 Yes \[u2074] 25791 T}@T{ 25792 Yes 25793 T}@T{ 25794 Yes 25795 T}@T{ 25796 No 25797 T}@T{ 25798 No 25799 T}@T{ 25800 Yes 25801 T}@T{ 25802 No 25803 T}@T{ 25804 No 25805 T}@T{ 25806 Yes 25807 T}@T{ 25808 Yes 25809 T} 25810 T{ 25811 Sia 25812 T}@T{ 25813 No 25814 T}@T{ 25815 No 25816 T}@T{ 25817 No 25818 T}@T{ 25819 No 25820 T}@T{ 25821 No 25822 T}@T{ 25823 No 25824 T}@T{ 25825 Yes 25826 T}@T{ 25827 No 25828 T}@T{ 25829 No 25830 T}@T{ 25831 No 25832 T}@T{ 25833 Yes 25834 T} 25835 T{ 25836 SMB 25837 T}@T{ 25838 No 25839 T}@T{ 25840 No 25841 T}@T{ 25842 Yes 25843 T}@T{ 25844 Yes 25845 T}@T{ 25846 No 25847 T}@T{ 25848 No 25849 T}@T{ 25850 Yes 25851 T}@T{ 25852 Yes 25853 T}@T{ 25854 No 25855 T}@T{ 25856 No 25857 T}@T{ 25858 Yes 25859 T} 25860 T{ 25861 SugarSync 25862 T}@T{ 25863 Yes 25864 T}@T{ 25865 Yes 25866 T}@T{ 25867 Yes 25868 T}@T{ 25869 Yes 25870 T}@T{ 25871 No 25872 T}@T{ 25873 No 25874 T}@T{ 25875 Yes 25876 T}@T{ 25877 No 25878 T}@T{ 25879 Yes 25880 T}@T{ 25881 No 25882 T}@T{ 25883 Yes 25884 T} 25885 T{ 25886 Storj 25887 T}@T{ 25888 Yes \[S2] 25889 T}@T{ 25890 Yes 25891 T}@T{ 25892 Yes 25893 T}@T{ 25894 No 25895 T}@T{ 25896 No 25897 T}@T{ 25898 Yes 25899 T}@T{ 25900 Yes 25901 T}@T{ 25902 No 25903 T}@T{ 25904 Yes 25905 T}@T{ 25906 No 25907 T}@T{ 25908 No 25909 T} 25910 T{ 25911 Uptobox 25912 T}@T{ 25913 No 25914 T}@T{ 25915 Yes 25916 T}@T{ 25917 Yes 25918 T}@T{ 25919 Yes 25920 T}@T{ 25921 No 25922 T}@T{ 25923 No 25924 T}@T{ 25925 No 25926 T}@T{ 25927 No 25928 T}@T{ 25929 No 25930 T}@T{ 25931 No 25932 T}@T{ 25933 No 25934 T} 25935 T{ 25936 WebDAV 25937 T}@T{ 25938 Yes 25939 T}@T{ 25940 Yes 25941 T}@T{ 25942 Yes 25943 T}@T{ 25944 Yes 25945 T}@T{ 25946 No 25947 T}@T{ 25948 No 25949 T}@T{ 25950 Yes \[S3] 25951 T}@T{ 25952 No 25953 T}@T{ 25954 No 25955 T}@T{ 25956 Yes 25957 T}@T{ 25958 Yes 25959 T} 25960 T{ 25961 Yandex Disk 25962 T}@T{ 25963 Yes 25964 T}@T{ 25965 Yes 25966 T}@T{ 25967 Yes 25968 T}@T{ 25969 Yes 25970 T}@T{ 25971 Yes 25972 T}@T{ 25973 No 25974 T}@T{ 25975 Yes 25976 T}@T{ 25977 No 25978 T}@T{ 25979 Yes 25980 T}@T{ 25981 Yes 25982 T}@T{ 25983 Yes 25984 T} 25985 T{ 25986 Zoho WorkDrive 25987 T}@T{ 25988 Yes 25989 T}@T{ 25990 Yes 25991 T}@T{ 25992 Yes 25993 T}@T{ 25994 Yes 25995 T}@T{ 25996 No 25997 T}@T{ 25998 No 25999 T}@T{ 26000 No 26001 T}@T{ 26002 No 26003 T}@T{ 26004 No 26005 T}@T{ 26006 Yes 26007 T}@T{ 26008 Yes 26009 T} 26010 T{ 26011 The local filesystem 26012 T}@T{ 26013 No 26014 T}@T{ 26015 No 26016 T}@T{ 26017 Yes 26018 T}@T{ 26019 Yes 26020 T}@T{ 26021 No 26022 T}@T{ 26023 No 26024 T}@T{ 26025 Yes 26026 T}@T{ 26027 Yes 26028 T}@T{ 26029 No 26030 T}@T{ 26031 Yes 26032 T}@T{ 26033 Yes 26034 T} 26035 .TE 26036 .PP 26037 \[S1] Note Swift implements this in order to delete directory markers 26038 but it doesn\[aq]t actually have a quicker way of deleting files other 26039 than deleting them individually. 26040 .PP 26041 \[S2] Storj implements this efficiently only for entire buckets. 26042 If purging a directory inside a bucket, files are deleted individually. 26043 .PP 26044 \[S3] StreamUpload is not supported with Nextcloud 26045 .PP 26046 \[u2074] Use the \f[C]--sftp-copy-is-hardlink\f[R] flag to enable. 26047 .PP 26048 \[u2075] Use the \f[C]--onedrive-delta\f[R] flag to enable. 26049 .SS Purge 26050 .PP 26051 This deletes a directory quicker than just deleting all the files in the 26052 directory. 26053 .SS Copy 26054 .PP 26055 Used when copying an object to and from the same remote. 26056 This known as a server-side copy so you can copy a file without 26057 downloading it and uploading it again. 26058 It is used if you use \f[C]rclone copy\f[R] or \f[C]rclone move\f[R] if 26059 the remote doesn\[aq]t support \f[C]Move\f[R] directly. 26060 .PP 26061 If the server doesn\[aq]t support \f[C]Copy\f[R] directly then for copy 26062 operations the file is downloaded then re-uploaded. 26063 .SS Move 26064 .PP 26065 Used when moving/renaming an object on the same remote. 26066 This is known as a server-side move of a file. 26067 This is used in \f[C]rclone move\f[R] if the server doesn\[aq]t support 26068 \f[C]DirMove\f[R]. 26069 .PP 26070 If the server isn\[aq]t capable of \f[C]Move\f[R] then rclone simulates 26071 it with \f[C]Copy\f[R] then delete. 26072 If the server doesn\[aq]t support \f[C]Copy\f[R] then rclone will 26073 download the file and re-upload it. 26074 .SS DirMove 26075 .PP 26076 This is used to implement \f[C]rclone move\f[R] to move a directory if 26077 possible. 26078 If it isn\[aq]t then it will use \f[C]Move\f[R] on each file (which 26079 falls back to \f[C]Copy\f[R] then download and upload - see 26080 \f[C]Move\f[R] section). 26081 .SS CleanUp 26082 .PP 26083 This is used for emptying the trash for a remote by 26084 \f[C]rclone cleanup\f[R]. 26085 .PP 26086 If the server can\[aq]t do \f[C]CleanUp\f[R] then 26087 \f[C]rclone cleanup\f[R] will return an error. 26088 .PP 26089 \[dd]\[dd] Note that while Box implements this it has to delete every 26090 file individually so it will be slower than emptying the trash via the 26091 WebUI 26092 .SS ListR 26093 .PP 26094 The remote supports a recursive list to list all the contents beneath a 26095 directory quickly. 26096 This enables the \f[C]--fast-list\f[R] flag to work. 26097 See the rclone docs (https://rclone.org/docs/#fast-list) for more 26098 details. 26099 .SS StreamUpload 26100 .PP 26101 Some remotes allow files to be uploaded without knowing the file size in 26102 advance. 26103 This allows certain operations to work without spooling the file to 26104 local disk first, e.g. 26105 \f[C]rclone rcat\f[R]. 26106 .SS MultithreadUpload 26107 .PP 26108 Some remotes allow transfers to the remote to be sent as chunks in 26109 parallel. 26110 If this is supported then rclone will use multi-thread copying to 26111 transfer files much faster. 26112 .SS LinkSharing 26113 .PP 26114 Sets the necessary permissions on a file or folder and prints a link 26115 that allows others to access them, even if they don\[aq]t have an 26116 account on the particular cloud provider. 26117 .SS About 26118 .PP 26119 Rclone \f[C]about\f[R] prints quota information for a remote. 26120 Typical output includes bytes used, free, quota and in trash. 26121 .PP 26122 If a remote lacks about capability \f[C]rclone about remote:\f[R]returns 26123 an error. 26124 .PP 26125 Backends without about capability cannot determine free space for an 26126 rclone mount, or use policy \f[C]mfs\f[R] (most free space) as a member 26127 of an rclone union remote. 26128 .PP 26129 See rclone about command (https://rclone.org/commands/rclone_about/) 26130 .SS EmptyDir 26131 .PP 26132 The remote supports empty directories. 26133 See Limitations (https://rclone.org/bugs/#limitations) for details. 26134 Most Object/Bucket-based remotes do not support this. 26135 .SH Global Flags 26136 .PP 26137 This describes the global flags available to every rclone command split 26138 into groups. 26139 .SS Copy 26140 .PP 26141 Flags for anything which can Copy a file. 26142 .IP 26143 .nf 26144 \f[C] 26145 --check-first Do all the checks before starting transfers 26146 -c, --checksum Check for changes with size & checksum (if available, or fallback to size only). 26147 --compare-dest stringArray Include additional comma separated server-side paths during comparison 26148 --copy-dest stringArray Implies --compare-dest but also copies files from paths into destination 26149 --cutoff-mode HARD|SOFT|CAUTIOUS Mode to stop transfers when reaching the max transfer limit HARD|SOFT|CAUTIOUS (default HARD) 26150 --ignore-case-sync Ignore case when synchronizing 26151 --ignore-checksum Skip post copy check of checksums 26152 --ignore-existing Skip all files that exist on destination 26153 --ignore-size Ignore size when skipping use modtime or checksum 26154 -I, --ignore-times Don\[aq]t skip items that match size and time - transfer all unconditionally 26155 --immutable Do not modify files, fail if existing files have been modified 26156 --inplace Download directly to destination file instead of atomic download to temp/rename 26157 --max-backlog int Maximum number of objects in sync or check backlog (default 10000) 26158 --max-duration Duration Maximum duration rclone will transfer data for (default 0s) 26159 --max-transfer SizeSuffix Maximum size of data to transfer (default off) 26160 -M, --metadata If set, preserve metadata when copying objects 26161 --modify-window Duration Max time diff to be considered the same (default 1ns) 26162 --multi-thread-chunk-size SizeSuffix Chunk size for multi-thread downloads / uploads, if not set by filesystem (default 64Mi) 26163 --multi-thread-cutoff SizeSuffix Use multi-thread downloads for files above this size (default 256Mi) 26164 --multi-thread-streams int Number of streams to use for multi-thread downloads (default 4) 26165 --multi-thread-write-buffer-size SizeSuffix In memory buffer size for writing when in multi-thread mode (default 128Ki) 26166 --no-check-dest Don\[aq]t check the destination, copy regardless 26167 --no-traverse Don\[aq]t traverse destination file system on copy 26168 --no-update-dir-modtime Don\[aq]t update directory modification times 26169 --no-update-modtime Don\[aq]t update destination modtime if files identical 26170 --order-by string Instructions on how to order the transfers, e.g. \[aq]size,descending\[aq] 26171 --partial-suffix string Add partial-suffix to temporary file name when --inplace is not used (default \[dq].partial\[dq]) 26172 --refresh-times Refresh the modtime of remote files 26173 --server-side-across-configs Allow server-side operations (e.g. copy) to work across different configs 26174 --size-only Skip based on size only, not modtime or checksum 26175 --streaming-upload-cutoff SizeSuffix Cutoff for switching to chunked upload if file size is unknown, upload starts after reaching cutoff or when file ends (default 100Ki) 26176 -u, --update Skip files that are newer on the destination 26177 \f[R] 26178 .fi 26179 .SS Sync 26180 .PP 26181 Flags just used for \f[C]rclone sync\f[R]. 26182 .IP 26183 .nf 26184 \f[C] 26185 --backup-dir string Make backups into hierarchy based in DIR 26186 --delete-after When synchronizing, delete files on destination after transferring (default) 26187 --delete-before When synchronizing, delete files on destination before transferring 26188 --delete-during When synchronizing, delete files during transfer 26189 --fix-case Force rename of case insensitive dest to match source 26190 --ignore-errors Delete even if there are I/O errors 26191 --max-delete int When synchronizing, limit the number of deletes (default -1) 26192 --max-delete-size SizeSuffix When synchronizing, limit the total size of deletes (default off) 26193 --suffix string Suffix to add to changed files 26194 --suffix-keep-extension Preserve the extension when using --suffix 26195 --track-renames When synchronizing, track file renames and do a server-side move if possible 26196 --track-renames-strategy string Strategies to use when synchronizing using track-renames hash|modtime|leaf (default \[dq]hash\[dq]) 26197 \f[R] 26198 .fi 26199 .SS Important 26200 .PP 26201 Important flags useful for most commands. 26202 .IP 26203 .nf 26204 \f[C] 26205 -n, --dry-run Do a trial run with no permanent changes 26206 -i, --interactive Enable interactive mode 26207 -v, --verbose count Print lots more stuff (repeat for more) 26208 \f[R] 26209 .fi 26210 .SS Check 26211 .PP 26212 Flags used for \f[C]rclone check\f[R]. 26213 .IP 26214 .nf 26215 \f[C] 26216 --max-backlog int Maximum number of objects in sync or check backlog (default 10000) 26217 \f[R] 26218 .fi 26219 .SS Networking 26220 .PP 26221 General networking and HTTP stuff. 26222 .IP 26223 .nf 26224 \f[C] 26225 --bind string Local address to bind to for outgoing connections, IPv4, IPv6 or name 26226 --bwlimit BwTimetable Bandwidth limit in KiB/s, or use suffix B|K|M|G|T|P or a full timetable 26227 --bwlimit-file BwTimetable Bandwidth limit per file in KiB/s, or use suffix B|K|M|G|T|P or a full timetable 26228 --ca-cert stringArray CA certificate used to verify servers 26229 --client-cert string Client SSL certificate (PEM) for mutual TLS auth 26230 --client-key string Client SSL private key (PEM) for mutual TLS auth 26231 --contimeout Duration Connect timeout (default 1m0s) 26232 --disable-http-keep-alives Disable HTTP keep-alives and use each connection once. 26233 --disable-http2 Disable HTTP/2 in the global transport 26234 --dscp string Set DSCP value to connections, value or name, e.g. CS1, LE, DF, AF21 26235 --expect-continue-timeout Duration Timeout when using expect / 100-continue in HTTP (default 1s) 26236 --header stringArray Set HTTP header for all transactions 26237 --header-download stringArray Set HTTP header for download transactions 26238 --header-upload stringArray Set HTTP header for upload transactions 26239 --no-check-certificate Do not verify the server SSL certificate (insecure) 26240 --no-gzip-encoding Don\[aq]t set Accept-Encoding: gzip 26241 --timeout Duration IO idle timeout (default 5m0s) 26242 --tpslimit float Limit HTTP transactions per second to this 26243 --tpslimit-burst int Max burst of transactions for --tpslimit (default 1) 26244 --use-cookies Enable session cookiejar 26245 --user-agent string Set the user-agent to a specified string (default \[dq]rclone/v1.66.0\[dq]) 26246 \f[R] 26247 .fi 26248 .SS Performance 26249 .PP 26250 Flags helpful for increasing performance. 26251 .IP 26252 .nf 26253 \f[C] 26254 --buffer-size SizeSuffix In memory buffer size when reading files for each --transfer (default 16Mi) 26255 --checkers int Number of checkers to run in parallel (default 8) 26256 --transfers int Number of file transfers to run in parallel (default 4) 26257 \f[R] 26258 .fi 26259 .SS Config 26260 .PP 26261 General configuration of rclone. 26262 .IP 26263 .nf 26264 \f[C] 26265 --ask-password Allow prompt for password for encrypted configuration (default true) 26266 --auto-confirm If enabled, do not request console confirmation 26267 --cache-dir string Directory rclone will use for caching (default \[dq]$HOME/.cache/rclone\[dq]) 26268 --color AUTO|NEVER|ALWAYS When to show colors (and other ANSI codes) AUTO|NEVER|ALWAYS (default AUTO) 26269 --config string Config file (default \[dq]$HOME/.config/artpar/artpar.conf\[dq]) 26270 --default-time Time Time to show if modtime is unknown for files and directories (default 2000-01-01T00:00:00Z) 26271 --disable string Disable a comma separated list of features (use --disable help to see a list) 26272 -n, --dry-run Do a trial run with no permanent changes 26273 --error-on-no-transfer Sets exit code 9 if no files are transferred, useful in scripts 26274 --fs-cache-expire-duration Duration Cache remotes for this long (0 to disable caching) (default 5m0s) 26275 --fs-cache-expire-interval Duration Interval to check for expired remotes (default 1m0s) 26276 --human-readable Print numbers in a human-readable format, sizes with suffix Ki|Mi|Gi|Ti|Pi 26277 -i, --interactive Enable interactive mode 26278 --kv-lock-time Duration Maximum time to keep key-value database locked by process (default 1s) 26279 --low-level-retries int Number of low level retries to do (default 10) 26280 --no-console Hide console window (supported on Windows only) 26281 --no-unicode-normalization Don\[aq]t normalize unicode characters in filenames 26282 --password-command SpaceSepList Command for supplying password for encrypted configuration 26283 --retries int Retry operations this many times if they fail (default 3) 26284 --retries-sleep Duration Interval between retrying operations if they fail, e.g. 500ms, 60s, 5m (0 to disable) (default 0s) 26285 --temp-dir string Directory rclone will use for temporary files (default \[dq]/tmp\[dq]) 26286 --use-mmap Use mmap allocator (see docs) 26287 --use-server-modtime Use server modified time instead of object metadata 26288 \f[R] 26289 .fi 26290 .SS Debugging 26291 .PP 26292 Flags for developers. 26293 .IP 26294 .nf 26295 \f[C] 26296 --cpuprofile string Write cpu profile to file 26297 --dump DumpFlags List of items to dump from: headers, bodies, requests, responses, auth, filters, goroutines, openfiles, mapper 26298 --dump-bodies Dump HTTP headers and bodies - may contain sensitive info 26299 --dump-headers Dump HTTP headers - may contain sensitive info 26300 --memprofile string Write memory profile to file 26301 \f[R] 26302 .fi 26303 .SS Filter 26304 .PP 26305 Flags for filtering directory listings. 26306 .IP 26307 .nf 26308 \f[C] 26309 --delete-excluded Delete files on dest excluded from sync 26310 --exclude stringArray Exclude files matching pattern 26311 --exclude-from stringArray Read file exclude patterns from file (use - to read from stdin) 26312 --exclude-if-present stringArray Exclude directories if filename is present 26313 --files-from stringArray Read list of source-file names from file (use - to read from stdin) 26314 --files-from-raw stringArray Read list of source-file names from file without any processing of lines (use - to read from stdin) 26315 -f, --filter stringArray Add a file filtering rule 26316 --filter-from stringArray Read file filtering patterns from a file (use - to read from stdin) 26317 --ignore-case Ignore case in filters (case insensitive) 26318 --include stringArray Include files matching pattern 26319 --include-from stringArray Read file include patterns from file (use - to read from stdin) 26320 --max-age Duration Only transfer files younger than this in s or suffix ms|s|m|h|d|w|M|y (default off) 26321 --max-depth int If set limits the recursion depth to this (default -1) 26322 --max-size SizeSuffix Only transfer files smaller than this in KiB or suffix B|K|M|G|T|P (default off) 26323 --metadata-exclude stringArray Exclude metadatas matching pattern 26324 --metadata-exclude-from stringArray Read metadata exclude patterns from file (use - to read from stdin) 26325 --metadata-filter stringArray Add a metadata filtering rule 26326 --metadata-filter-from stringArray Read metadata filtering patterns from a file (use - to read from stdin) 26327 --metadata-include stringArray Include metadatas matching pattern 26328 --metadata-include-from stringArray Read metadata include patterns from file (use - to read from stdin) 26329 --min-age Duration Only transfer files older than this in s or suffix ms|s|m|h|d|w|M|y (default off) 26330 --min-size SizeSuffix Only transfer files bigger than this in KiB or suffix B|K|M|G|T|P (default off) 26331 \f[R] 26332 .fi 26333 .SS Listing 26334 .PP 26335 Flags for listing directories. 26336 .IP 26337 .nf 26338 \f[C] 26339 --default-time Time Time to show if modtime is unknown for files and directories (default 2000-01-01T00:00:00Z) 26340 --fast-list Use recursive list if available; uses more memory but fewer transactions 26341 \f[R] 26342 .fi 26343 .SS Logging 26344 .PP 26345 Logging and statistics. 26346 .IP 26347 .nf 26348 \f[C] 26349 --log-file string Log everything to this file 26350 --log-format string Comma separated list of log format options (default \[dq]date,time\[dq]) 26351 --log-level LogLevel Log level DEBUG|INFO|NOTICE|ERROR (default NOTICE) 26352 --log-systemd Activate systemd integration for the logger 26353 --max-stats-groups int Maximum number of stats groups to keep in memory, on max oldest is discarded (default 1000) 26354 -P, --progress Show progress during transfer 26355 --progress-terminal-title Show progress on the terminal title (requires -P/--progress) 26356 -q, --quiet Print as little stuff as possible 26357 --stats Duration Interval between printing stats, e.g. 500ms, 60s, 5m (0 to disable) (default 1m0s) 26358 --stats-file-name-length int Max file name length in stats (0 for no limit) (default 45) 26359 --stats-log-level LogLevel Log level to show --stats output DEBUG|INFO|NOTICE|ERROR (default INFO) 26360 --stats-one-line Make the stats fit on one line 26361 --stats-one-line-date Enable --stats-one-line and add current date/time prefix 26362 --stats-one-line-date-format string Enable --stats-one-line-date and use custom formatted date: Enclose date string in double quotes (\[dq]), see https://golang.org/pkg/time/#Time.Format 26363 --stats-unit string Show data rate in stats as either \[aq]bits\[aq] or \[aq]bytes\[aq] per second (default \[dq]bytes\[dq]) 26364 --syslog Use Syslog for logging 26365 --syslog-facility string Facility for syslog, e.g. KERN,USER,... (default \[dq]DAEMON\[dq]) 26366 --use-json-log Use json log format 26367 -v, --verbose count Print lots more stuff (repeat for more) 26368 \f[R] 26369 .fi 26370 .SS Metadata 26371 .PP 26372 Flags to control metadata. 26373 .IP 26374 .nf 26375 \f[C] 26376 -M, --metadata If set, preserve metadata when copying objects 26377 --metadata-exclude stringArray Exclude metadatas matching pattern 26378 --metadata-exclude-from stringArray Read metadata exclude patterns from file (use - to read from stdin) 26379 --metadata-filter stringArray Add a metadata filtering rule 26380 --metadata-filter-from stringArray Read metadata filtering patterns from a file (use - to read from stdin) 26381 --metadata-include stringArray Include metadatas matching pattern 26382 --metadata-include-from stringArray Read metadata include patterns from file (use - to read from stdin) 26383 --metadata-mapper SpaceSepList Program to run to transforming metadata before upload 26384 --metadata-set stringArray Add metadata key=value when uploading 26385 \f[R] 26386 .fi 26387 .SS RC 26388 .PP 26389 Flags to control the Remote Control API. 26390 .IP 26391 .nf 26392 \f[C] 26393 --rc Enable the remote control server 26394 --rc-addr stringArray IPaddress:Port or :Port to bind server to (default [localhost:5572]) 26395 --rc-allow-origin string Origin which cross-domain request (CORS) can be executed from 26396 --rc-baseurl string Prefix for URLs - leave blank for root 26397 --rc-cert string TLS PEM key (concatenation of certificate and CA certificate) 26398 --rc-client-ca string Client certificate authority to verify clients with 26399 --rc-enable-metrics Enable prometheus metrics on /metrics 26400 --rc-files string Path to local files to serve on the HTTP server 26401 --rc-htpasswd string A htpasswd file - if not provided no authentication is done 26402 --rc-job-expire-duration Duration Expire finished async jobs older than this value (default 1m0s) 26403 --rc-job-expire-interval Duration Interval to check for expired async jobs (default 10s) 26404 --rc-key string TLS PEM Private key 26405 --rc-max-header-bytes int Maximum size of request header (default 4096) 26406 --rc-min-tls-version string Minimum TLS version that is acceptable (default \[dq]tls1.0\[dq]) 26407 --rc-no-auth Don\[aq]t require auth for certain methods 26408 --rc-pass string Password for authentication 26409 --rc-realm string Realm for authentication 26410 --rc-salt string Password hashing salt (default \[dq]dlPL2MqE\[dq]) 26411 --rc-serve Enable the serving of remote objects 26412 --rc-server-read-timeout Duration Timeout for server reading data (default 1h0m0s) 26413 --rc-server-write-timeout Duration Timeout for server writing data (default 1h0m0s) 26414 --rc-template string User-specified template 26415 --rc-user string User name for authentication 26416 --rc-web-fetch-url string URL to fetch the releases for webgui (default \[dq]https://api.github.com/repos/artpar/artpar-webui-react/releases/latest\[dq]) 26417 --rc-web-gui Launch WebGUI on localhost 26418 --rc-web-gui-force-update Force update to latest version of web gui 26419 --rc-web-gui-no-open-browser Don\[aq]t open the browser automatically 26420 --rc-web-gui-update Check and update to latest version of web gui 26421 \f[R] 26422 .fi 26423 .SS Backend 26424 .PP 26425 Backend only flags. 26426 These can be set in the config file also. 26427 .IP 26428 .nf 26429 \f[C] 26430 --alias-description string Description of the remote 26431 --alias-remote string Remote or path to alias 26432 --azureblob-access-tier string Access tier of blob: hot, cool, cold or archive 26433 --azureblob-account string Azure Storage Account Name 26434 --azureblob-archive-tier-delete Delete archive tier blobs before overwriting 26435 --azureblob-chunk-size SizeSuffix Upload chunk size (default 4Mi) 26436 --azureblob-client-certificate-password string Password for the certificate file (optional) (obscured) 26437 --azureblob-client-certificate-path string Path to a PEM or PKCS12 certificate file including the private key 26438 --azureblob-client-id string The ID of the client in use 26439 --azureblob-client-secret string One of the service principal\[aq]s client secrets 26440 --azureblob-client-send-certificate-chain Send the certificate chain when using certificate auth 26441 --azureblob-delete-snapshots string Set to specify how to deal with snapshots on blob deletion 26442 --azureblob-description string Description of the remote 26443 --azureblob-directory-markers Upload an empty object with a trailing slash when a new directory is created 26444 --azureblob-disable-checksum Don\[aq]t store MD5 checksum with object metadata 26445 --azureblob-encoding Encoding The encoding for the backend (default Slash,BackSlash,Del,Ctl,RightPeriod,InvalidUtf8) 26446 --azureblob-endpoint string Endpoint for the service 26447 --azureblob-env-auth Read credentials from runtime (environment variables, CLI or MSI) 26448 --azureblob-key string Storage Account Shared Key 26449 --azureblob-list-chunk int Size of blob list (default 5000) 26450 --azureblob-msi-client-id string Object ID of the user-assigned MSI to use, if any 26451 --azureblob-msi-mi-res-id string Azure resource ID of the user-assigned MSI to use, if any 26452 --azureblob-msi-object-id string Object ID of the user-assigned MSI to use, if any 26453 --azureblob-no-check-container If set, don\[aq]t attempt to check the container exists or create it 26454 --azureblob-no-head-object If set, do not do HEAD before GET when getting objects 26455 --azureblob-password string The user\[aq]s password (obscured) 26456 --azureblob-public-access string Public access level of a container: blob or container 26457 --azureblob-sas-url string SAS URL for container level access only 26458 --azureblob-service-principal-file string Path to file containing credentials for use with a service principal 26459 --azureblob-tenant string ID of the service principal\[aq]s tenant. Also called its directory ID 26460 --azureblob-upload-concurrency int Concurrency for multipart uploads (default 16) 26461 --azureblob-upload-cutoff string Cutoff for switching to chunked upload (<= 256 MiB) (deprecated) 26462 --azureblob-use-emulator Uses local storage emulator if provided as \[aq]true\[aq] 26463 --azureblob-use-msi Use a managed service identity to authenticate (only works in Azure) 26464 --azureblob-username string User name (usually an email address) 26465 --azurefiles-account string Azure Storage Account Name 26466 --azurefiles-chunk-size SizeSuffix Upload chunk size (default 4Mi) 26467 --azurefiles-client-certificate-password string Password for the certificate file (optional) (obscured) 26468 --azurefiles-client-certificate-path string Path to a PEM or PKCS12 certificate file including the private key 26469 --azurefiles-client-id string The ID of the client in use 26470 --azurefiles-client-secret string One of the service principal\[aq]s client secrets 26471 --azurefiles-client-send-certificate-chain Send the certificate chain when using certificate auth 26472 --azurefiles-connection-string string Azure Files Connection String 26473 --azurefiles-description string Description of the remote 26474 --azurefiles-encoding Encoding The encoding for the backend (default Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Del,Ctl,RightPeriod,InvalidUtf8,Dot) 26475 --azurefiles-endpoint string Endpoint for the service 26476 --azurefiles-env-auth Read credentials from runtime (environment variables, CLI or MSI) 26477 --azurefiles-key string Storage Account Shared Key 26478 --azurefiles-max-stream-size SizeSuffix Max size for streamed files (default 10Gi) 26479 --azurefiles-msi-client-id string Object ID of the user-assigned MSI to use, if any 26480 --azurefiles-msi-mi-res-id string Azure resource ID of the user-assigned MSI to use, if any 26481 --azurefiles-msi-object-id string Object ID of the user-assigned MSI to use, if any 26482 --azurefiles-password string The user\[aq]s password (obscured) 26483 --azurefiles-sas-url string SAS URL 26484 --azurefiles-service-principal-file string Path to file containing credentials for use with a service principal 26485 --azurefiles-share-name string Azure Files Share Name 26486 --azurefiles-tenant string ID of the service principal\[aq]s tenant. Also called its directory ID 26487 --azurefiles-upload-concurrency int Concurrency for multipart uploads (default 16) 26488 --azurefiles-use-msi Use a managed service identity to authenticate (only works in Azure) 26489 --azurefiles-username string User name (usually an email address) 26490 --b2-account string Account ID or Application Key ID 26491 --b2-chunk-size SizeSuffix Upload chunk size (default 96Mi) 26492 --b2-copy-cutoff SizeSuffix Cutoff for switching to multipart copy (default 4Gi) 26493 --b2-description string Description of the remote 26494 --b2-disable-checksum Disable checksums for large (> upload cutoff) files 26495 --b2-download-auth-duration Duration Time before the public link authorization token will expire in s or suffix ms|s|m|h|d (default 1w) 26496 --b2-download-url string Custom endpoint for downloads 26497 --b2-encoding Encoding The encoding for the backend (default Slash,BackSlash,Del,Ctl,InvalidUtf8,Dot) 26498 --b2-endpoint string Endpoint for the service 26499 --b2-hard-delete Permanently delete files on remote removal, otherwise hide files 26500 --b2-key string Application Key 26501 --b2-lifecycle int Set the number of days deleted files should be kept when creating a bucket 26502 --b2-test-mode string A flag string for X-Bz-Test-Mode header for debugging 26503 --b2-upload-concurrency int Concurrency for multipart uploads (default 4) 26504 --b2-upload-cutoff SizeSuffix Cutoff for switching to chunked upload (default 200Mi) 26505 --b2-version-at Time Show file versions as they were at the specified time (default off) 26506 --b2-versions Include old versions in directory listings 26507 --box-access-token string Box App Primary Access Token 26508 --box-auth-url string Auth server URL 26509 --box-box-config-file string Box App config.json location 26510 --box-box-sub-type string (default \[dq]user\[dq]) 26511 --box-client-id string OAuth Client Id 26512 --box-client-secret string OAuth Client Secret 26513 --box-commit-retries int Max number of times to try committing a multipart file (default 100) 26514 --box-description string Description of the remote 26515 --box-encoding Encoding The encoding for the backend (default Slash,BackSlash,Del,Ctl,RightSpace,InvalidUtf8,Dot) 26516 --box-impersonate string Impersonate this user ID when using a service account 26517 --box-list-chunk int Size of listing chunk 1-1000 (default 1000) 26518 --box-owned-by string Only show items owned by the login (email address) passed in 26519 --box-root-folder-id string Fill in for rclone to use a non root folder as its starting point 26520 --box-token string OAuth Access Token as a JSON blob 26521 --box-token-url string Token server url 26522 --box-upload-cutoff SizeSuffix Cutoff for switching to multipart upload (>= 50 MiB) (default 50Mi) 26523 --cache-chunk-clean-interval Duration How often should the cache perform cleanups of the chunk storage (default 1m0s) 26524 --cache-chunk-no-memory Disable the in-memory cache for storing chunks during streaming 26525 --cache-chunk-path string Directory to cache chunk files (default \[dq]$HOME/.cache/rclone/cache-backend\[dq]) 26526 --cache-chunk-size SizeSuffix The size of a chunk (partial file data) (default 5Mi) 26527 --cache-chunk-total-size SizeSuffix The total size that the chunks can take up on the local disk (default 10Gi) 26528 --cache-db-path string Directory to store file structure metadata DB (default \[dq]$HOME/.cache/rclone/cache-backend\[dq]) 26529 --cache-db-purge Clear all the cached data for this remote on start 26530 --cache-db-wait-time Duration How long to wait for the DB to be available - 0 is unlimited (default 1s) 26531 --cache-description string Description of the remote 26532 --cache-info-age Duration How long to cache file structure information (directory listings, file size, times, etc.) (default 6h0m0s) 26533 --cache-plex-insecure string Skip all certificate verification when connecting to the Plex server 26534 --cache-plex-password string The password of the Plex user (obscured) 26535 --cache-plex-url string The URL of the Plex server 26536 --cache-plex-username string The username of the Plex user 26537 --cache-read-retries int How many times to retry a read from a cache storage (default 10) 26538 --cache-remote string Remote to cache 26539 --cache-rps int Limits the number of requests per second to the source FS (-1 to disable) (default -1) 26540 --cache-tmp-upload-path string Directory to keep temporary files until they are uploaded 26541 --cache-tmp-wait-time Duration How long should files be stored in local cache before being uploaded (default 15s) 26542 --cache-workers int How many workers should run in parallel to download chunks (default 4) 26543 --cache-writes Cache file data on writes through the FS 26544 --chunker-chunk-size SizeSuffix Files larger than chunk size will be split in chunks (default 2Gi) 26545 --chunker-description string Description of the remote 26546 --chunker-fail-hard Choose how chunker should handle files with missing or invalid chunks 26547 --chunker-hash-type string Choose how chunker handles hash sums (default \[dq]md5\[dq]) 26548 --chunker-remote string Remote to chunk/unchunk 26549 --combine-description string Description of the remote 26550 --combine-upstreams SpaceSepList Upstreams for combining 26551 --compress-description string Description of the remote 26552 --compress-level int GZIP compression level (-2 to 9) (default -1) 26553 --compress-mode string Compression mode (default \[dq]gzip\[dq]) 26554 --compress-ram-cache-limit SizeSuffix Some remotes don\[aq]t allow the upload of files with unknown size (default 20Mi) 26555 --compress-remote string Remote to compress 26556 -L, --copy-links Follow symlinks and copy the pointed to item 26557 --crypt-description string Description of the remote 26558 --crypt-directory-name-encryption Option to either encrypt directory names or leave them intact (default true) 26559 --crypt-filename-encoding string How to encode the encrypted filename to text string (default \[dq]base32\[dq]) 26560 --crypt-filename-encryption string How to encrypt the filenames (default \[dq]standard\[dq]) 26561 --crypt-no-data-encryption Option to either encrypt file data or leave it unencrypted 26562 --crypt-pass-bad-blocks If set this will pass bad blocks through as all 0 26563 --crypt-password string Password or pass phrase for encryption (obscured) 26564 --crypt-password2 string Password or pass phrase for salt (obscured) 26565 --crypt-remote string Remote to encrypt/decrypt 26566 --crypt-server-side-across-configs Deprecated: use --server-side-across-configs instead 26567 --crypt-show-mapping For all files listed show how the names encrypt 26568 --crypt-strict-names If set, this will raise an error when crypt comes across a filename that can\[aq]t be decrypted 26569 --crypt-suffix string If this is set it will override the default suffix of \[dq].bin\[dq] (default \[dq].bin\[dq]) 26570 --drive-acknowledge-abuse Set to allow files which return cannotDownloadAbusiveFile to be downloaded 26571 --drive-allow-import-name-change Allow the filetype to change when uploading Google docs 26572 --drive-auth-owner-only Only consider files owned by the authenticated user 26573 --drive-auth-url string Auth server URL 26574 --drive-chunk-size SizeSuffix Upload chunk size (default 8Mi) 26575 --drive-client-id string Google Application Client Id 26576 --drive-client-secret string OAuth Client Secret 26577 --drive-copy-shortcut-content Server side copy contents of shortcuts instead of the shortcut 26578 --drive-description string Description of the remote 26579 --drive-disable-http2 Disable drive using http2 (default true) 26580 --drive-encoding Encoding The encoding for the backend (default InvalidUtf8) 26581 --drive-env-auth Get IAM credentials from runtime (environment variables or instance meta data if no env vars) 26582 --drive-export-formats string Comma separated list of preferred formats for downloading Google docs (default \[dq]docx,xlsx,pptx,svg\[dq]) 26583 --drive-fast-list-bug-fix Work around a bug in Google Drive listing (default true) 26584 --drive-formats string Deprecated: See export_formats 26585 --drive-impersonate string Impersonate this user when using a service account 26586 --drive-import-formats string Comma separated list of preferred formats for uploading Google docs 26587 --drive-keep-revision-forever Keep new head revision of each file forever 26588 --drive-list-chunk int Size of listing chunk 100-1000, 0 to disable (default 1000) 26589 --drive-metadata-labels Bits Control whether labels should be read or written in metadata (default off) 26590 --drive-metadata-owner Bits Control whether owner should be read or written in metadata (default read) 26591 --drive-metadata-permissions Bits Control whether permissions should be read or written in metadata (default off) 26592 --drive-pacer-burst int Number of API calls to allow without sleeping (default 100) 26593 --drive-pacer-min-sleep Duration Minimum time to sleep between API calls (default 100ms) 26594 --drive-resource-key string Resource key for accessing a link-shared file 26595 --drive-root-folder-id string ID of the root folder 26596 --drive-scope string Comma separated list of scopes that rclone should use when requesting access from drive 26597 --drive-server-side-across-configs Deprecated: use --server-side-across-configs instead 26598 --drive-service-account-credentials string Service Account Credentials JSON blob 26599 --drive-service-account-file string Service Account Credentials JSON file path 26600 --drive-shared-with-me Only show files that are shared with me 26601 --drive-show-all-gdocs Show all Google Docs including non-exportable ones in listings 26602 --drive-size-as-quota Show sizes as storage quota usage, not actual size 26603 --drive-skip-checksum-gphotos Skip checksums on Google photos and videos only 26604 --drive-skip-dangling-shortcuts If set skip dangling shortcut files 26605 --drive-skip-gdocs Skip google documents in all listings 26606 --drive-skip-shortcuts If set skip shortcut files 26607 --drive-starred-only Only show files that are starred 26608 --drive-stop-on-download-limit Make download limit errors be fatal 26609 --drive-stop-on-upload-limit Make upload limit errors be fatal 26610 --drive-team-drive string ID of the Shared Drive (Team Drive) 26611 --drive-token string OAuth Access Token as a JSON blob 26612 --drive-token-url string Token server url 26613 --drive-trashed-only Only show files that are in the trash 26614 --drive-upload-cutoff SizeSuffix Cutoff for switching to chunked upload (default 8Mi) 26615 --drive-use-created-date Use file created date instead of modified date 26616 --drive-use-shared-date Use date file was shared instead of modified date 26617 --drive-use-trash Send files to the trash instead of deleting permanently (default true) 26618 --drive-v2-download-min-size SizeSuffix If Object\[aq]s are greater, use drive v2 API to download (default off) 26619 --dropbox-auth-url string Auth server URL 26620 --dropbox-batch-commit-timeout Duration Max time to wait for a batch to finish committing (default 10m0s) 26621 --dropbox-batch-mode string Upload file batching sync|async|off (default \[dq]sync\[dq]) 26622 --dropbox-batch-size int Max number of files in upload batch 26623 --dropbox-batch-timeout Duration Max time to allow an idle upload batch before uploading (default 0s) 26624 --dropbox-chunk-size SizeSuffix Upload chunk size (< 150Mi) (default 48Mi) 26625 --dropbox-client-id string OAuth Client Id 26626 --dropbox-client-secret string OAuth Client Secret 26627 --dropbox-description string Description of the remote 26628 --dropbox-encoding Encoding The encoding for the backend (default Slash,BackSlash,Del,RightSpace,InvalidUtf8,Dot) 26629 --dropbox-impersonate string Impersonate this user when using a business account 26630 --dropbox-pacer-min-sleep Duration Minimum time to sleep between API calls (default 10ms) 26631 --dropbox-shared-files Instructs rclone to work on individual shared files 26632 --dropbox-shared-folders Instructs rclone to work on shared folders 26633 --dropbox-token string OAuth Access Token as a JSON blob 26634 --dropbox-token-url string Token server url 26635 --fichier-api-key string Your API Key, get it from https://1fichier.com/console/params.pl 26636 --fichier-cdn Set if you wish to use CDN download links 26637 --fichier-description string Description of the remote 26638 --fichier-encoding Encoding The encoding for the backend (default Slash,LtGt,DoubleQuote,SingleQuote,BackQuote,Dollar,BackSlash,Del,Ctl,LeftSpace,RightSpace,InvalidUtf8,Dot) 26639 --fichier-file-password string If you want to download a shared file that is password protected, add this parameter (obscured) 26640 --fichier-folder-password string If you want to list the files in a shared folder that is password protected, add this parameter (obscured) 26641 --fichier-shared-folder string If you want to download a shared folder, add this parameter 26642 --filefabric-description string Description of the remote 26643 --filefabric-encoding Encoding The encoding for the backend (default Slash,Del,Ctl,InvalidUtf8,Dot) 26644 --filefabric-permanent-token string Permanent Authentication Token 26645 --filefabric-root-folder-id string ID of the root folder 26646 --filefabric-token string Session Token 26647 --filefabric-token-expiry string Token expiry time 26648 --filefabric-url string URL of the Enterprise File Fabric to connect to 26649 --filefabric-version string Version read from the file fabric 26650 --ftp-ask-password Allow asking for FTP password when needed 26651 --ftp-close-timeout Duration Maximum time to wait for a response to close (default 1m0s) 26652 --ftp-concurrency int Maximum number of FTP simultaneous connections, 0 for unlimited 26653 --ftp-description string Description of the remote 26654 --ftp-disable-epsv Disable using EPSV even if server advertises support 26655 --ftp-disable-mlsd Disable using MLSD even if server advertises support 26656 --ftp-disable-tls13 Disable TLS 1.3 (workaround for FTP servers with buggy TLS) 26657 --ftp-disable-utf8 Disable using UTF-8 even if server advertises support 26658 --ftp-encoding Encoding The encoding for the backend (default Slash,Del,Ctl,RightSpace,Dot) 26659 --ftp-explicit-tls Use Explicit FTPS (FTP over TLS) 26660 --ftp-force-list-hidden Use LIST -a to force listing of hidden files and folders. This will disable the use of MLSD 26661 --ftp-host string FTP host to connect to 26662 --ftp-idle-timeout Duration Max time before closing idle connections (default 1m0s) 26663 --ftp-no-check-certificate Do not verify the TLS certificate of the server 26664 --ftp-pass string FTP password (obscured) 26665 --ftp-port int FTP port number (default 21) 26666 --ftp-shut-timeout Duration Maximum time to wait for data connection closing status (default 1m0s) 26667 --ftp-socks-proxy string Socks 5 proxy host 26668 --ftp-tls Use Implicit FTPS (FTP over TLS) 26669 --ftp-tls-cache-size int Size of TLS session cache for all control and data connections (default 32) 26670 --ftp-user string FTP username (default \[dq]$USER\[dq]) 26671 --ftp-writing-mdtm Use MDTM to set modification time (VsFtpd quirk) 26672 --gcs-anonymous Access public buckets and objects without credentials 26673 --gcs-auth-url string Auth server URL 26674 --gcs-bucket-acl string Access Control List for new buckets 26675 --gcs-bucket-policy-only Access checks should use bucket-level IAM policies 26676 --gcs-client-id string OAuth Client Id 26677 --gcs-client-secret string OAuth Client Secret 26678 --gcs-decompress If set this will decompress gzip encoded objects 26679 --gcs-description string Description of the remote 26680 --gcs-directory-markers Upload an empty object with a trailing slash when a new directory is created 26681 --gcs-encoding Encoding The encoding for the backend (default Slash,CrLf,InvalidUtf8,Dot) 26682 --gcs-endpoint string Endpoint for the service 26683 --gcs-env-auth Get GCP IAM credentials from runtime (environment variables or instance meta data if no env vars) 26684 --gcs-location string Location for the newly created buckets 26685 --gcs-no-check-bucket If set, don\[aq]t attempt to check the bucket exists or create it 26686 --gcs-object-acl string Access Control List for new objects 26687 --gcs-project-number string Project number 26688 --gcs-service-account-file string Service Account Credentials JSON file path 26689 --gcs-storage-class string The storage class to use when storing objects in Google Cloud Storage 26690 --gcs-token string OAuth Access Token as a JSON blob 26691 --gcs-token-url string Token server url 26692 --gcs-user-project string User project 26693 --gphotos-auth-url string Auth server URL 26694 --gphotos-batch-commit-timeout Duration Max time to wait for a batch to finish committing (default 10m0s) 26695 --gphotos-batch-mode string Upload file batching sync|async|off (default \[dq]sync\[dq]) 26696 --gphotos-batch-size int Max number of files in upload batch 26697 --gphotos-batch-timeout Duration Max time to allow an idle upload batch before uploading (default 0s) 26698 --gphotos-client-id string OAuth Client Id 26699 --gphotos-client-secret string OAuth Client Secret 26700 --gphotos-description string Description of the remote 26701 --gphotos-encoding Encoding The encoding for the backend (default Slash,CrLf,InvalidUtf8,Dot) 26702 --gphotos-include-archived Also view and download archived media 26703 --gphotos-read-only Set to make the Google Photos backend read only 26704 --gphotos-read-size Set to read the size of media items 26705 --gphotos-start-year int Year limits the photos to be downloaded to those which are uploaded after the given year (default 2000) 26706 --gphotos-token string OAuth Access Token as a JSON blob 26707 --gphotos-token-url string Token server url 26708 --hasher-auto-size SizeSuffix Auto-update checksum for files smaller than this size (disabled by default) 26709 --hasher-description string Description of the remote 26710 --hasher-hashes CommaSepList Comma separated list of supported checksum types (default md5,sha1) 26711 --hasher-max-age Duration Maximum time to keep checksums in cache (0 = no cache, off = cache forever) (default off) 26712 --hasher-remote string Remote to cache checksums for (e.g. myRemote:path) 26713 --hdfs-data-transfer-protection string Kerberos data transfer protection: authentication|integrity|privacy 26714 --hdfs-description string Description of the remote 26715 --hdfs-encoding Encoding The encoding for the backend (default Slash,Colon,Del,Ctl,InvalidUtf8,Dot) 26716 --hdfs-namenode CommaSepList Hadoop name nodes and ports 26717 --hdfs-service-principal-name string Kerberos service principal name for the namenode 26718 --hdfs-username string Hadoop user name 26719 --hidrive-auth-url string Auth server URL 26720 --hidrive-chunk-size SizeSuffix Chunksize for chunked uploads (default 48Mi) 26721 --hidrive-client-id string OAuth Client Id 26722 --hidrive-client-secret string OAuth Client Secret 26723 --hidrive-description string Description of the remote 26724 --hidrive-disable-fetching-member-count Do not fetch number of objects in directories unless it is absolutely necessary 26725 --hidrive-encoding Encoding The encoding for the backend (default Slash,Dot) 26726 --hidrive-endpoint string Endpoint for the service (default \[dq]https://api.hidrive.strato.com/2.1\[dq]) 26727 --hidrive-root-prefix string The root/parent folder for all paths (default \[dq]/\[dq]) 26728 --hidrive-scope-access string Access permissions that rclone should use when requesting access from HiDrive (default \[dq]rw\[dq]) 26729 --hidrive-scope-role string User-level that rclone should use when requesting access from HiDrive (default \[dq]user\[dq]) 26730 --hidrive-token string OAuth Access Token as a JSON blob 26731 --hidrive-token-url string Token server url 26732 --hidrive-upload-concurrency int Concurrency for chunked uploads (default 4) 26733 --hidrive-upload-cutoff SizeSuffix Cutoff/Threshold for chunked uploads (default 96Mi) 26734 --http-description string Description of the remote 26735 --http-headers CommaSepList Set HTTP headers for all transactions 26736 --http-no-head Don\[aq]t use HEAD requests 26737 --http-no-slash Set this if the site doesn\[aq]t end directories with / 26738 --http-url string URL of HTTP host to connect to 26739 --imagekit-description string Description of the remote 26740 --imagekit-encoding Encoding The encoding for the backend (default Slash,LtGt,DoubleQuote,Dollar,Question,Hash,Percent,BackSlash,Del,Ctl,InvalidUtf8,Dot,SquareBracket) 26741 --imagekit-endpoint string You can find your ImageKit.io URL endpoint in your [dashboard](https://imagekit.io/dashboard/developer/api-keys) 26742 --imagekit-only-signed Restrict unsigned image URLs If you have configured Restrict unsigned image URLs in your dashboard settings, set this to true 26743 --imagekit-private-key string You can find your ImageKit.io private key in your [dashboard](https://imagekit.io/dashboard/developer/api-keys) 26744 --imagekit-public-key string You can find your ImageKit.io public key in your [dashboard](https://imagekit.io/dashboard/developer/api-keys) 26745 --imagekit-upload-tags string Tags to add to the uploaded files, e.g. \[dq]tag1,tag2\[dq] 26746 --imagekit-versions Include old versions in directory listings 26747 --internetarchive-access-key-id string IAS3 Access Key 26748 --internetarchive-description string Description of the remote 26749 --internetarchive-disable-checksum Don\[aq]t ask the server to test against MD5 checksum calculated by rclone (default true) 26750 --internetarchive-encoding Encoding The encoding for the backend (default Slash,LtGt,CrLf,Del,Ctl,InvalidUtf8,Dot) 26751 --internetarchive-endpoint string IAS3 Endpoint (default \[dq]https://s3.us.archive.org\[dq]) 26752 --internetarchive-front-endpoint string Host of InternetArchive Frontend (default \[dq]https://archive.org\[dq]) 26753 --internetarchive-secret-access-key string IAS3 Secret Key (password) 26754 --internetarchive-wait-archive Duration Timeout for waiting the server\[aq]s processing tasks (specifically archive and book_op) to finish (default 0s) 26755 --jottacloud-auth-url string Auth server URL 26756 --jottacloud-client-id string OAuth Client Id 26757 --jottacloud-client-secret string OAuth Client Secret 26758 --jottacloud-description string Description of the remote 26759 --jottacloud-encoding Encoding The encoding for the backend (default Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,Del,Ctl,InvalidUtf8,Dot) 26760 --jottacloud-hard-delete Delete files permanently rather than putting them into the trash 26761 --jottacloud-md5-memory-limit SizeSuffix Files bigger than this will be cached on disk to calculate the MD5 if required (default 10Mi) 26762 --jottacloud-no-versions Avoid server side versioning by deleting files and recreating files instead of overwriting them 26763 --jottacloud-token string OAuth Access Token as a JSON blob 26764 --jottacloud-token-url string Token server url 26765 --jottacloud-trashed-only Only show files that are in the trash 26766 --jottacloud-upload-resume-limit SizeSuffix Files bigger than this can be resumed if the upload fail\[aq]s (default 10Mi) 26767 --koofr-description string Description of the remote 26768 --koofr-encoding Encoding The encoding for the backend (default Slash,BackSlash,Del,Ctl,InvalidUtf8,Dot) 26769 --koofr-endpoint string The Koofr API endpoint to use 26770 --koofr-mountid string Mount ID of the mount to use 26771 --koofr-password string Your password for rclone (generate one at https://app.koofr.net/app/admin/preferences/password) (obscured) 26772 --koofr-provider string Choose your storage provider 26773 --koofr-setmtime Does the backend support setting modification time (default true) 26774 --koofr-user string Your user name 26775 --linkbox-description string Description of the remote 26776 --linkbox-token string Token from https://www.linkbox.to/admin/account 26777 -l, --links Translate symlinks to/from regular files with a \[aq].rclonelink\[aq] extension 26778 --local-case-insensitive Force the filesystem to report itself as case insensitive 26779 --local-case-sensitive Force the filesystem to report itself as case sensitive 26780 --local-description string Description of the remote 26781 --local-encoding Encoding The encoding for the backend (default Slash,Dot) 26782 --local-no-check-updated Don\[aq]t check to see if the files change during upload 26783 --local-no-preallocate Disable preallocation of disk space for transferred files 26784 --local-no-set-modtime Disable setting modtime 26785 --local-no-sparse Disable sparse files for multi-thread downloads 26786 --local-nounc Disable UNC (long path names) conversion on Windows 26787 --local-unicode-normalization Apply unicode NFC normalization to paths and filenames 26788 --local-zero-size-links Assume the Stat size of links is zero (and read them instead) (deprecated) 26789 --mailru-auth-url string Auth server URL 26790 --mailru-check-hash What should copy do if file checksum is mismatched or invalid (default true) 26791 --mailru-client-id string OAuth Client Id 26792 --mailru-client-secret string OAuth Client Secret 26793 --mailru-description string Description of the remote 26794 --mailru-encoding Encoding The encoding for the backend (default Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Del,Ctl,InvalidUtf8,Dot) 26795 --mailru-pass string Password (obscured) 26796 --mailru-speedup-enable Skip full upload if there is another file with same data hash (default true) 26797 --mailru-speedup-file-patterns string Comma separated list of file name patterns eligible for speedup (put by hash) (default \[dq]*.mkv,*.avi,*.mp4,*.mp3,*.zip,*.gz,*.rar,*.pdf\[dq]) 26798 --mailru-speedup-max-disk SizeSuffix This option allows you to disable speedup (put by hash) for large files (default 3Gi) 26799 --mailru-speedup-max-memory SizeSuffix Files larger than the size given below will always be hashed on disk (default 32Mi) 26800 --mailru-token string OAuth Access Token as a JSON blob 26801 --mailru-token-url string Token server url 26802 --mailru-user string User name (usually email) 26803 --mega-debug Output more debug from Mega 26804 --mega-description string Description of the remote 26805 --mega-encoding Encoding The encoding for the backend (default Slash,InvalidUtf8,Dot) 26806 --mega-hard-delete Delete files permanently rather than putting them into the trash 26807 --mega-pass string Password (obscured) 26808 --mega-use-https Use HTTPS for transfers 26809 --mega-user string User name 26810 --memory-description string Description of the remote 26811 --netstorage-account string Set the NetStorage account name 26812 --netstorage-description string Description of the remote 26813 --netstorage-host string Domain+path of NetStorage host to connect to 26814 --netstorage-protocol string Select between HTTP or HTTPS protocol (default \[dq]https\[dq]) 26815 --netstorage-secret string Set the NetStorage account secret/G2O key for authentication (obscured) 26816 -x, --one-file-system Don\[aq]t cross filesystem boundaries (unix/macOS only) 26817 --onedrive-access-scopes SpaceSepList Set scopes to be requested by rclone (default Files.Read Files.ReadWrite Files.Read.All Files.ReadWrite.All Sites.Read.All offline_access) 26818 --onedrive-auth-url string Auth server URL 26819 --onedrive-av-override Allows download of files the server thinks has a virus 26820 --onedrive-chunk-size SizeSuffix Chunk size to upload files with - must be multiple of 320k (327,680 bytes) (default 10Mi) 26821 --onedrive-client-id string OAuth Client Id 26822 --onedrive-client-secret string OAuth Client Secret 26823 --onedrive-delta If set rclone will use delta listing to implement recursive listings 26824 --onedrive-description string Description of the remote 26825 --onedrive-drive-id string The ID of the drive to use 26826 --onedrive-drive-type string The type of the drive (personal | business | documentLibrary) 26827 --onedrive-encoding Encoding The encoding for the backend (default Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Del,Ctl,LeftSpace,LeftTilde,RightSpace,RightPeriod,InvalidUtf8,Dot) 26828 --onedrive-expose-onenote-files Set to make OneNote files show up in directory listings 26829 --onedrive-hash-type string Specify the hash in use for the backend (default \[dq]auto\[dq]) 26830 --onedrive-link-password string Set the password for links created by the link command 26831 --onedrive-link-scope string Set the scope of the links created by the link command (default \[dq]anonymous\[dq]) 26832 --onedrive-link-type string Set the type of the links created by the link command (default \[dq]view\[dq]) 26833 --onedrive-list-chunk int Size of listing chunk (default 1000) 26834 --onedrive-metadata-permissions Bits Control whether permissions should be read or written in metadata (default off) 26835 --onedrive-no-versions Remove all versions on modifying operations 26836 --onedrive-region string Choose national cloud region for OneDrive (default \[dq]global\[dq]) 26837 --onedrive-root-folder-id string ID of the root folder 26838 --onedrive-server-side-across-configs Deprecated: use --server-side-across-configs instead 26839 --onedrive-token string OAuth Access Token as a JSON blob 26840 --onedrive-token-url string Token server url 26841 --oos-attempt-resume-upload If true attempt to resume previously started multipart upload for the object 26842 --oos-chunk-size SizeSuffix Chunk size to use for uploading (default 5Mi) 26843 --oos-compartment string Object storage compartment OCID 26844 --oos-config-file string Path to OCI config file (default \[dq]\[ti]/.oci/config\[dq]) 26845 --oos-config-profile string Profile name inside the oci config file (default \[dq]Default\[dq]) 26846 --oos-copy-cutoff SizeSuffix Cutoff for switching to multipart copy (default 4.656Gi) 26847 --oos-copy-timeout Duration Timeout for copy (default 1m0s) 26848 --oos-description string Description of the remote 26849 --oos-disable-checksum Don\[aq]t store MD5 checksum with object metadata 26850 --oos-encoding Encoding The encoding for the backend (default Slash,InvalidUtf8,Dot) 26851 --oos-endpoint string Endpoint for Object storage API 26852 --oos-leave-parts-on-error If true avoid calling abort upload on a failure, leaving all successfully uploaded parts for manual recovery 26853 --oos-max-upload-parts int Maximum number of parts in a multipart upload (default 10000) 26854 --oos-namespace string Object storage namespace 26855 --oos-no-check-bucket If set, don\[aq]t attempt to check the bucket exists or create it 26856 --oos-provider string Choose your Auth Provider (default \[dq]env_auth\[dq]) 26857 --oos-region string Object storage Region 26858 --oos-sse-customer-algorithm string If using SSE-C, the optional header that specifies \[dq]AES256\[dq] as the encryption algorithm 26859 --oos-sse-customer-key string To use SSE-C, the optional header that specifies the base64-encoded 256-bit encryption key to use to 26860 --oos-sse-customer-key-file string To use SSE-C, a file containing the base64-encoded string of the AES-256 encryption key associated 26861 --oos-sse-customer-key-sha256 string If using SSE-C, The optional header that specifies the base64-encoded SHA256 hash of the encryption 26862 --oos-sse-kms-key-id string if using your own master key in vault, this header specifies the 26863 --oos-storage-tier string The storage class to use when storing new objects in storage. https://docs.oracle.com/en-us/iaas/Content/Object/Concepts/understandingstoragetiers.htm (default \[dq]Standard\[dq]) 26864 --oos-upload-concurrency int Concurrency for multipart uploads (default 10) 26865 --oos-upload-cutoff SizeSuffix Cutoff for switching to chunked upload (default 200Mi) 26866 --opendrive-chunk-size SizeSuffix Files will be uploaded in chunks this size (default 10Mi) 26867 --opendrive-description string Description of the remote 26868 --opendrive-encoding Encoding The encoding for the backend (default Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,LeftSpace,LeftCrLfHtVt,RightSpace,RightCrLfHtVt,InvalidUtf8,Dot) 26869 --opendrive-password string Password (obscured) 26870 --opendrive-username string Username 26871 --pcloud-auth-url string Auth server URL 26872 --pcloud-client-id string OAuth Client Id 26873 --pcloud-client-secret string OAuth Client Secret 26874 --pcloud-description string Description of the remote 26875 --pcloud-encoding Encoding The encoding for the backend (default Slash,BackSlash,Del,Ctl,InvalidUtf8,Dot) 26876 --pcloud-hostname string Hostname to connect to (default \[dq]api.pcloud.com\[dq]) 26877 --pcloud-password string Your pcloud password (obscured) 26878 --pcloud-root-folder-id string Fill in for rclone to use a non root folder as its starting point (default \[dq]d0\[dq]) 26879 --pcloud-token string OAuth Access Token as a JSON blob 26880 --pcloud-token-url string Token server url 26881 --pcloud-username string Your pcloud username 26882 --pikpak-auth-url string Auth server URL 26883 --pikpak-client-id string OAuth Client Id 26884 --pikpak-client-secret string OAuth Client Secret 26885 --pikpak-description string Description of the remote 26886 --pikpak-encoding Encoding The encoding for the backend (default Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Ctl,LeftSpace,RightSpace,RightPeriod,InvalidUtf8,Dot) 26887 --pikpak-hash-memory-limit SizeSuffix Files bigger than this will be cached on disk to calculate hash if required (default 10Mi) 26888 --pikpak-pass string Pikpak password (obscured) 26889 --pikpak-root-folder-id string ID of the root folder 26890 --pikpak-token string OAuth Access Token as a JSON blob 26891 --pikpak-token-url string Token server url 26892 --pikpak-trashed-only Only show files that are in the trash 26893 --pikpak-use-trash Send files to the trash instead of deleting permanently (default true) 26894 --pikpak-user string Pikpak username 26895 --premiumizeme-auth-url string Auth server URL 26896 --premiumizeme-client-id string OAuth Client Id 26897 --premiumizeme-client-secret string OAuth Client Secret 26898 --premiumizeme-description string Description of the remote 26899 --premiumizeme-encoding Encoding The encoding for the backend (default Slash,DoubleQuote,BackSlash,Del,Ctl,InvalidUtf8,Dot) 26900 --premiumizeme-token string OAuth Access Token as a JSON blob 26901 --premiumizeme-token-url string Token server url 26902 --protondrive-2fa string The 2FA code 26903 --protondrive-app-version string The app version string (default \[dq]macos-drive\[at]1.0.0-alpha.1+rclone\[dq]) 26904 --protondrive-description string Description of the remote 26905 --protondrive-enable-caching Caches the files and folders metadata to reduce API calls (default true) 26906 --protondrive-encoding Encoding The encoding for the backend (default Slash,LeftSpace,RightSpace,InvalidUtf8,Dot) 26907 --protondrive-mailbox-password string The mailbox password of your two-password proton account (obscured) 26908 --protondrive-original-file-size Return the file size before encryption (default true) 26909 --protondrive-password string The password of your proton account (obscured) 26910 --protondrive-replace-existing-draft Create a new revision when filename conflict is detected 26911 --protondrive-username string The username of your proton account 26912 --putio-auth-url string Auth server URL 26913 --putio-client-id string OAuth Client Id 26914 --putio-client-secret string OAuth Client Secret 26915 --putio-description string Description of the remote 26916 --putio-encoding Encoding The encoding for the backend (default Slash,BackSlash,Del,Ctl,InvalidUtf8,Dot) 26917 --putio-token string OAuth Access Token as a JSON blob 26918 --putio-token-url string Token server url 26919 --qingstor-access-key-id string QingStor Access Key ID 26920 --qingstor-chunk-size SizeSuffix Chunk size to use for uploading (default 4Mi) 26921 --qingstor-connection-retries int Number of connection retries (default 3) 26922 --qingstor-description string Description of the remote 26923 --qingstor-encoding Encoding The encoding for the backend (default Slash,Ctl,InvalidUtf8) 26924 --qingstor-endpoint string Enter an endpoint URL to connection QingStor API 26925 --qingstor-env-auth Get QingStor credentials from runtime 26926 --qingstor-secret-access-key string QingStor Secret Access Key (password) 26927 --qingstor-upload-concurrency int Concurrency for multipart uploads (default 1) 26928 --qingstor-upload-cutoff SizeSuffix Cutoff for switching to chunked upload (default 200Mi) 26929 --qingstor-zone string Zone to connect to 26930 --quatrix-api-key string API key for accessing Quatrix account 26931 --quatrix-description string Description of the remote 26932 --quatrix-effective-upload-time string Wanted upload time for one chunk (default \[dq]4s\[dq]) 26933 --quatrix-encoding Encoding The encoding for the backend (default Slash,BackSlash,Del,Ctl,InvalidUtf8,Dot) 26934 --quatrix-hard-delete Delete files permanently rather than putting them into the trash 26935 --quatrix-host string Host name of Quatrix account 26936 --quatrix-maximal-summary-chunk-size SizeSuffix The maximal summary for all chunks. It should not be less than \[aq]transfers\[aq]*\[aq]minimal_chunk_size\[aq] (default 95.367Mi) 26937 --quatrix-minimal-chunk-size SizeSuffix The minimal size for one chunk (default 9.537Mi) 26938 --quatrix-skip-project-folders Skip project folders in operations 26939 --s3-access-key-id string AWS Access Key ID 26940 --s3-acl string Canned ACL used when creating buckets and storing or copying objects 26941 --s3-bucket-acl string Canned ACL used when creating buckets 26942 --s3-chunk-size SizeSuffix Chunk size to use for uploading (default 5Mi) 26943 --s3-copy-cutoff SizeSuffix Cutoff for switching to multipart copy (default 4.656Gi) 26944 --s3-decompress If set this will decompress gzip encoded objects 26945 --s3-description string Description of the remote 26946 --s3-directory-markers Upload an empty object with a trailing slash when a new directory is created 26947 --s3-disable-checksum Don\[aq]t store MD5 checksum with object metadata 26948 --s3-disable-http2 Disable usage of http2 for S3 backends 26949 --s3-download-url string Custom endpoint for downloads 26950 --s3-encoding Encoding The encoding for the backend (default Slash,InvalidUtf8,Dot) 26951 --s3-endpoint string Endpoint for S3 API 26952 --s3-env-auth Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars) 26953 --s3-force-path-style If true use path style access if false use virtual hosted style (default true) 26954 --s3-leave-parts-on-error If true avoid calling abort upload on a failure, leaving all successfully uploaded parts on S3 for manual recovery 26955 --s3-list-chunk int Size of listing chunk (response list for each ListObject S3 request) (default 1000) 26956 --s3-list-url-encode Tristate Whether to url encode listings: true/false/unset (default unset) 26957 --s3-list-version int Version of ListObjects to use: 1,2 or 0 for auto 26958 --s3-location-constraint string Location constraint - must be set to match the Region 26959 --s3-max-upload-parts int Maximum number of parts in a multipart upload (default 10000) 26960 --s3-might-gzip Tristate Set this if the backend might gzip objects (default unset) 26961 --s3-no-check-bucket If set, don\[aq]t attempt to check the bucket exists or create it 26962 --s3-no-head If set, don\[aq]t HEAD uploaded objects to check integrity 26963 --s3-no-head-object If set, do not do HEAD before GET when getting objects 26964 --s3-no-system-metadata Suppress setting and reading of system metadata 26965 --s3-profile string Profile to use in the shared credentials file 26966 --s3-provider string Choose your S3 provider 26967 --s3-region string Region to connect to 26968 --s3-requester-pays Enables requester pays option when interacting with S3 bucket 26969 --s3-secret-access-key string AWS Secret Access Key (password) 26970 --s3-server-side-encryption string The server-side encryption algorithm used when storing this object in S3 26971 --s3-session-token string An AWS session token 26972 --s3-shared-credentials-file string Path to the shared credentials file 26973 --s3-sse-customer-algorithm string If using SSE-C, the server-side encryption algorithm used when storing this object in S3 26974 --s3-sse-customer-key string To use SSE-C you may provide the secret encryption key used to encrypt/decrypt your data 26975 --s3-sse-customer-key-base64 string If using SSE-C you must provide the secret encryption key encoded in base64 format to encrypt/decrypt your data 26976 --s3-sse-customer-key-md5 string If using SSE-C you may provide the secret encryption key MD5 checksum (optional) 26977 --s3-sse-kms-key-id string If using KMS ID you must provide the ARN of Key 26978 --s3-storage-class string The storage class to use when storing new objects in S3 26979 --s3-sts-endpoint string Endpoint for STS 26980 --s3-upload-concurrency int Concurrency for multipart uploads and copies (default 4) 26981 --s3-upload-cutoff SizeSuffix Cutoff for switching to chunked upload (default 200Mi) 26982 --s3-use-accelerate-endpoint If true use the AWS S3 accelerated endpoint 26983 --s3-use-accept-encoding-gzip Accept-Encoding: gzip Whether to send Accept-Encoding: gzip header (default unset) 26984 --s3-use-already-exists Tristate Set if rclone should report BucketAlreadyExists errors on bucket creation (default unset) 26985 --s3-use-dual-stack If true use AWS S3 dual-stack endpoint (IPv6 support) 26986 --s3-use-multipart-etag Tristate Whether to use ETag in multipart uploads for verification (default unset) 26987 --s3-use-multipart-uploads Tristate Set if rclone should use multipart uploads (default unset) 26988 --s3-use-presigned-request Whether to use a presigned request or PutObject for single part uploads 26989 --s3-v2-auth If true use v2 authentication 26990 --s3-version-at Time Show file versions as they were at the specified time (default off) 26991 --s3-version-deleted Show deleted file markers when using versions 26992 --s3-versions Include old versions in directory listings 26993 --seafile-2fa Two-factor authentication (\[aq]true\[aq] if the account has 2FA enabled) 26994 --seafile-create-library Should rclone create a library if it doesn\[aq]t exist 26995 --seafile-description string Description of the remote 26996 --seafile-encoding Encoding The encoding for the backend (default Slash,DoubleQuote,BackSlash,Ctl,InvalidUtf8) 26997 --seafile-library string Name of the library 26998 --seafile-library-key string Library password (for encrypted libraries only) (obscured) 26999 --seafile-pass string Password (obscured) 27000 --seafile-url string URL of seafile host to connect to 27001 --seafile-user string User name (usually email address) 27002 --sftp-ask-password Allow asking for SFTP password when needed 27003 --sftp-chunk-size SizeSuffix Upload and download chunk size (default 32Ki) 27004 --sftp-ciphers SpaceSepList Space separated list of ciphers to be used for session encryption, ordered by preference 27005 --sftp-concurrency int The maximum number of outstanding requests for one file (default 64) 27006 --sftp-copy-is-hardlink Set to enable server side copies using hardlinks 27007 --sftp-description string Description of the remote 27008 --sftp-disable-concurrent-reads If set don\[aq]t use concurrent reads 27009 --sftp-disable-concurrent-writes If set don\[aq]t use concurrent writes 27010 --sftp-disable-hashcheck Disable the execution of SSH commands to determine if remote file hashing is available 27011 --sftp-host string SSH host to connect to 27012 --sftp-host-key-algorithms SpaceSepList Space separated list of host key algorithms, ordered by preference 27013 --sftp-idle-timeout Duration Max time before closing idle connections (default 1m0s) 27014 --sftp-key-exchange SpaceSepList Space separated list of key exchange algorithms, ordered by preference 27015 --sftp-key-file string Path to PEM-encoded private key file 27016 --sftp-key-file-pass string The passphrase to decrypt the PEM-encoded private key file (obscured) 27017 --sftp-key-pem string Raw PEM-encoded private key 27018 --sftp-key-use-agent When set forces the usage of the ssh-agent 27019 --sftp-known-hosts-file string Optional path to known_hosts file 27020 --sftp-macs SpaceSepList Space separated list of MACs (message authentication code) algorithms, ordered by preference 27021 --sftp-md5sum-command string The command used to read md5 hashes 27022 --sftp-pass string SSH password, leave blank to use ssh-agent (obscured) 27023 --sftp-path-override string Override path used by SSH shell commands 27024 --sftp-port int SSH port number (default 22) 27025 --sftp-pubkey-file string Optional path to public key file 27026 --sftp-server-command string Specifies the path or command to run a sftp server on the remote host 27027 --sftp-set-env SpaceSepList Environment variables to pass to sftp and commands 27028 --sftp-set-modtime Set the modified time on the remote if set (default true) 27029 --sftp-sha1sum-command string The command used to read sha1 hashes 27030 --sftp-shell-type string The type of SSH shell on remote server, if any 27031 --sftp-skip-links Set to skip any symlinks and any other non regular files 27032 --sftp-socks-proxy string Socks 5 proxy host 27033 --sftp-ssh SpaceSepList Path and arguments to external ssh binary 27034 --sftp-subsystem string Specifies the SSH2 subsystem on the remote host (default \[dq]sftp\[dq]) 27035 --sftp-use-fstat If set use fstat instead of stat 27036 --sftp-use-insecure-cipher Enable the use of insecure ciphers and key exchange methods 27037 --sftp-user string SSH username (default \[dq]$USER\[dq]) 27038 --sharefile-auth-url string Auth server URL 27039 --sharefile-chunk-size SizeSuffix Upload chunk size (default 64Mi) 27040 --sharefile-client-id string OAuth Client Id 27041 --sharefile-client-secret string OAuth Client Secret 27042 --sharefile-description string Description of the remote 27043 --sharefile-encoding Encoding The encoding for the backend (default Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Ctl,LeftSpace,LeftPeriod,RightSpace,RightPeriod,InvalidUtf8,Dot) 27044 --sharefile-endpoint string Endpoint for API calls 27045 --sharefile-root-folder-id string ID of the root folder 27046 --sharefile-token string OAuth Access Token as a JSON blob 27047 --sharefile-token-url string Token server url 27048 --sharefile-upload-cutoff SizeSuffix Cutoff for switching to multipart upload (default 128Mi) 27049 --sia-api-password string Sia Daemon API Password (obscured) 27050 --sia-api-url string Sia daemon API URL, like http://sia.daemon.host:9980 (default \[dq]http://127.0.0.1:9980\[dq]) 27051 --sia-description string Description of the remote 27052 --sia-encoding Encoding The encoding for the backend (default Slash,Question,Hash,Percent,Del,Ctl,InvalidUtf8,Dot) 27053 --sia-user-agent string Siad User Agent (default \[dq]Sia-Agent\[dq]) 27054 --skip-links Don\[aq]t warn about skipped symlinks 27055 --smb-case-insensitive Whether the server is configured to be case-insensitive (default true) 27056 --smb-description string Description of the remote 27057 --smb-domain string Domain name for NTLM authentication (default \[dq]WORKGROUP\[dq]) 27058 --smb-encoding Encoding The encoding for the backend (default Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Ctl,RightSpace,RightPeriod,InvalidUtf8,Dot) 27059 --smb-hide-special-share Hide special shares (e.g. print$) which users aren\[aq]t supposed to access (default true) 27060 --smb-host string SMB server hostname to connect to 27061 --smb-idle-timeout Duration Max time before closing idle connections (default 1m0s) 27062 --smb-pass string SMB password (obscured) 27063 --smb-port int SMB port number (default 445) 27064 --smb-spn string Service principal name 27065 --smb-user string SMB username (default \[dq]$USER\[dq]) 27066 --storj-access-grant string Access grant 27067 --storj-api-key string API key 27068 --storj-description string Description of the remote 27069 --storj-passphrase string Encryption passphrase 27070 --storj-provider string Choose an authentication method (default \[dq]existing\[dq]) 27071 --storj-satellite-address string Satellite address (default \[dq]us1.storj.io\[dq]) 27072 --sugarsync-access-key-id string Sugarsync Access Key ID 27073 --sugarsync-app-id string Sugarsync App ID 27074 --sugarsync-authorization string Sugarsync authorization 27075 --sugarsync-authorization-expiry string Sugarsync authorization expiry 27076 --sugarsync-deleted-id string Sugarsync deleted folder id 27077 --sugarsync-description string Description of the remote 27078 --sugarsync-encoding Encoding The encoding for the backend (default Slash,Ctl,InvalidUtf8,Dot) 27079 --sugarsync-hard-delete Permanently delete files if true 27080 --sugarsync-private-access-key string Sugarsync Private Access Key 27081 --sugarsync-refresh-token string Sugarsync refresh token 27082 --sugarsync-root-id string Sugarsync root id 27083 --sugarsync-user string Sugarsync user 27084 --swift-application-credential-id string Application Credential ID (OS_APPLICATION_CREDENTIAL_ID) 27085 --swift-application-credential-name string Application Credential Name (OS_APPLICATION_CREDENTIAL_NAME) 27086 --swift-application-credential-secret string Application Credential Secret (OS_APPLICATION_CREDENTIAL_SECRET) 27087 --swift-auth string Authentication URL for server (OS_AUTH_URL) 27088 --swift-auth-token string Auth Token from alternate authentication - optional (OS_AUTH_TOKEN) 27089 --swift-auth-version int AuthVersion - optional - set to (1,2,3) if your auth URL has no version (ST_AUTH_VERSION) 27090 --swift-chunk-size SizeSuffix Above this size files will be chunked into a _segments container (default 5Gi) 27091 --swift-description string Description of the remote 27092 --swift-domain string User domain - optional (v3 auth) (OS_USER_DOMAIN_NAME) 27093 --swift-encoding Encoding The encoding for the backend (default Slash,InvalidUtf8) 27094 --swift-endpoint-type string Endpoint type to choose from the service catalogue (OS_ENDPOINT_TYPE) (default \[dq]public\[dq]) 27095 --swift-env-auth Get swift credentials from environment variables in standard OpenStack form 27096 --swift-key string API key or password (OS_PASSWORD) 27097 --swift-leave-parts-on-error If true avoid calling abort upload on a failure 27098 --swift-no-chunk Don\[aq]t chunk files during streaming upload 27099 --swift-no-large-objects Disable support for static and dynamic large objects 27100 --swift-region string Region name - optional (OS_REGION_NAME) 27101 --swift-storage-policy string The storage policy to use when creating a new container 27102 --swift-storage-url string Storage URL - optional (OS_STORAGE_URL) 27103 --swift-tenant string Tenant name - optional for v1 auth, this or tenant_id required otherwise (OS_TENANT_NAME or OS_PROJECT_NAME) 27104 --swift-tenant-domain string Tenant domain - optional (v3 auth) (OS_PROJECT_DOMAIN_NAME) 27105 --swift-tenant-id string Tenant ID - optional for v1 auth, this or tenant required otherwise (OS_TENANT_ID) 27106 --swift-user string User name to log in (OS_USERNAME) 27107 --swift-user-id string User ID to log in - optional - most swift systems use user and leave this blank (v3 auth) (OS_USER_ID) 27108 --union-action-policy string Policy to choose upstream on ACTION category (default \[dq]epall\[dq]) 27109 --union-cache-time int Cache time of usage and free space (in seconds) (default 120) 27110 --union-create-policy string Policy to choose upstream on CREATE category (default \[dq]epmfs\[dq]) 27111 --union-description string Description of the remote 27112 --union-min-free-space SizeSuffix Minimum viable free space for lfs/eplfs policies (default 1Gi) 27113 --union-search-policy string Policy to choose upstream on SEARCH category (default \[dq]ff\[dq]) 27114 --union-upstreams string List of space separated upstreams 27115 --uptobox-access-token string Your access token 27116 --uptobox-description string Description of the remote 27117 --uptobox-encoding Encoding The encoding for the backend (default Slash,LtGt,DoubleQuote,BackQuote,Del,Ctl,LeftSpace,InvalidUtf8,Dot) 27118 --uptobox-private Set to make uploaded files private 27119 --webdav-bearer-token string Bearer token instead of user/pass (e.g. a Macaroon) 27120 --webdav-bearer-token-command string Command to run to get a bearer token 27121 --webdav-description string Description of the remote 27122 --webdav-encoding string The encoding for the backend 27123 --webdav-headers CommaSepList Set HTTP headers for all transactions 27124 --webdav-nextcloud-chunk-size SizeSuffix Nextcloud upload chunk size (default 10Mi) 27125 --webdav-owncloud-exclude-shares Exclude ownCloud shares 27126 --webdav-pacer-min-sleep Duration Minimum time to sleep between API calls (default 10ms) 27127 --webdav-pass string Password (obscured) 27128 --webdav-url string URL of http host to connect to 27129 --webdav-user string User name 27130 --webdav-vendor string Name of the WebDAV site/service/software you are using 27131 --yandex-auth-url string Auth server URL 27132 --yandex-client-id string OAuth Client Id 27133 --yandex-client-secret string OAuth Client Secret 27134 --yandex-description string Description of the remote 27135 --yandex-encoding Encoding The encoding for the backend (default Slash,Del,Ctl,InvalidUtf8,Dot) 27136 --yandex-hard-delete Delete files permanently rather than putting them into the trash 27137 --yandex-token string OAuth Access Token as a JSON blob 27138 --yandex-token-url string Token server url 27139 --zoho-auth-url string Auth server URL 27140 --zoho-client-id string OAuth Client Id 27141 --zoho-client-secret string OAuth Client Secret 27142 --zoho-description string Description of the remote 27143 --zoho-encoding Encoding The encoding for the backend (default Del,Ctl,InvalidUtf8) 27144 --zoho-region string Zoho region to connect to 27145 --zoho-token string OAuth Access Token as a JSON blob 27146 --zoho-token-url string Token server url 27147 \f[R] 27148 .fi 27149 .SH Docker Volume Plugin 27150 .SS Introduction 27151 .PP 27152 Docker 1.9 has added support for creating named 27153 volumes (https://docs.docker.com/storage/volumes/) via command-line 27154 interface (https://docs.docker.com/engine/reference/commandline/volume_create/) 27155 and mounting them in containers as a way to share data between them. 27156 Since Docker 1.10 you can create named volumes with Docker 27157 Compose (https://docs.docker.com/compose/) by descriptions in 27158 docker-compose.yml (https://docs.docker.com/compose/compose-file/compose-file-v2/#volume-configuration-reference) 27159 files for use by container groups on a single host. 27160 As of Docker 1.12 volumes are supported by Docker 27161 Swarm (https://docs.docker.com/engine/swarm/key-concepts/) included with 27162 Docker Engine and created from descriptions in swarm compose 27163 v3 (https://docs.docker.com/compose/compose-file/compose-file-v3/#volume-configuration-reference) 27164 files for use with \f[I]swarm stacks\f[R] across multiple cluster nodes. 27165 .PP 27166 Docker Volume 27167 Plugins (https://docs.docker.com/engine/extend/plugins_volume/) augment 27168 the default \f[C]local\f[R] volume driver included in Docker with 27169 stateful volumes shared across containers and hosts. 27170 Unlike local volumes, your data will \f[I]not\f[R] be deleted when such 27171 volume is removed. 27172 Plugins can run managed by the docker daemon, as a native system service 27173 (under systemd, \f[I]sysv\f[R] or \f[I]upstart\f[R]) or as a standalone 27174 executable. 27175 Rclone can run as docker volume plugin in all these modes. 27176 It interacts with the local docker daemon via plugin 27177 API (https://docs.docker.com/engine/extend/plugin_api/) and handles 27178 mounting of remote file systems into docker containers so it must run on 27179 the same host as the docker daemon or on every Swarm node. 27180 .SS Getting started 27181 .PP 27182 In the first example we will use the SFTP (https://rclone.org/sftp/) 27183 rclone volume with Docker engine on a standalone Ubuntu machine. 27184 .PP 27185 Start from installing Docker (https://docs.docker.com/engine/install/) 27186 on the host. 27187 .PP 27188 The \f[I]FUSE\f[R] driver is a prerequisite for rclone mounting and 27189 should be installed on host: 27190 .IP 27191 .nf 27192 \f[C] 27193 sudo apt-get -y install fuse 27194 \f[R] 27195 .fi 27196 .PP 27197 Create two directories required by rclone docker plugin: 27198 .IP 27199 .nf 27200 \f[C] 27201 sudo mkdir -p /var/lib/docker-plugins/rclone/config 27202 sudo mkdir -p /var/lib/docker-plugins/rclone/cache 27203 \f[R] 27204 .fi 27205 .PP 27206 Install the managed rclone docker plugin for your architecture (here 27207 \f[C]amd64\f[R]): 27208 .IP 27209 .nf 27210 \f[C] 27211 docker plugin install rclone/docker-volume-rclone:amd64 args=\[dq]-v\[dq] --alias rclone --grant-all-permissions 27212 docker plugin list 27213 \f[R] 27214 .fi 27215 .PP 27216 Create your SFTP volume (https://rclone.org/sftp/#standard-options): 27217 .IP 27218 .nf 27219 \f[C] 27220 docker volume create firstvolume -d rclone -o type=sftp -o sftp-host=_hostname_ -o sftp-user=_username_ -o sftp-pass=_password_ -o allow-other=true 27221 \f[R] 27222 .fi 27223 .PP 27224 Note that since all options are static, you don\[aq]t even have to run 27225 \f[C]rclone config\f[R] or create the \f[C]rclone.conf\f[R] file (but 27226 the \f[C]config\f[R] directory should still be present). 27227 In the simplest case you can use \f[C]localhost\f[R] as 27228 \f[I]hostname\f[R] and your SSH credentials as \f[I]username\f[R] and 27229 \f[I]password\f[R]. 27230 You can also change the remote path to your home directory on the host, 27231 for example \f[C]-o path=/home/username\f[R]. 27232 .PP 27233 Time to create a test container and mount the volume into it: 27234 .IP 27235 .nf 27236 \f[C] 27237 docker run --rm -it -v firstvolume:/mnt --workdir /mnt ubuntu:latest bash 27238 \f[R] 27239 .fi 27240 .PP 27241 If all goes well, you will enter the new container and change right to 27242 the mounted SFTP remote. 27243 You can type \f[C]ls\f[R] to list the mounted directory or otherwise 27244 play with it. 27245 Type \f[C]exit\f[R] when you are done. 27246 The container will stop but the volume will stay, ready to be reused. 27247 When it\[aq]s not needed anymore, remove it: 27248 .IP 27249 .nf 27250 \f[C] 27251 docker volume list 27252 docker volume remove firstvolume 27253 \f[R] 27254 .fi 27255 .PP 27256 Now let us try \f[B]something more elaborate\f[R]: Google 27257 Drive (https://rclone.org/drive/) volume on multi-node Docker Swarm. 27258 .PP 27259 You should start from installing Docker and FUSE, creating plugin 27260 directories and installing rclone plugin on \f[I]every\f[R] swarm node. 27261 Then setup the Swarm (https://docs.docker.com/engine/swarm/swarm-mode/). 27262 .PP 27263 Google Drive volumes need an access token which can be setup via web 27264 browser and will be periodically renewed by rclone. 27265 The managed plugin cannot run a browser so we will use a technique 27266 similar to the rclone setup on a headless 27267 box (https://rclone.org/remote_setup/). 27268 .PP 27269 Run rclone config (https://rclone.org/commands/rclone_config_create/) on 27270 \f[I]another\f[R] machine equipped with \f[I]web browser\f[R] and 27271 graphical user interface. 27272 Create the Google Drive 27273 remote (https://rclone.org/drive/#standard-options). 27274 When done, transfer the resulting \f[C]rclone.conf\f[R] to the Swarm 27275 cluster and save as 27276 \f[C]/var/lib/docker-plugins/rclone/config/rclone.conf\f[R] on 27277 \f[I]every\f[R] node. 27278 By default this location is accessible only to the root user so you will 27279 need appropriate privileges. 27280 The resulting config will look like this: 27281 .IP 27282 .nf 27283 \f[C] 27284 [gdrive] 27285 type = drive 27286 scope = drive 27287 drive_id = 1234567... 27288 root_folder_id = 0Abcd... 27289 token = {\[dq]access_token\[dq]:...} 27290 \f[R] 27291 .fi 27292 .PP 27293 Now create the file named \f[C]example.yml\f[R] with a swarm stack 27294 description like this: 27295 .IP 27296 .nf 27297 \f[C] 27298 version: \[aq]3\[aq] 27299 services: 27300 heimdall: 27301 image: linuxserver/heimdall:latest 27302 ports: [8080:80] 27303 volumes: [configdata:/config] 27304 volumes: 27305 configdata: 27306 driver: rclone 27307 driver_opts: 27308 remote: \[aq]gdrive:heimdall\[aq] 27309 allow_other: \[aq]true\[aq] 27310 vfs_cache_mode: full 27311 poll_interval: 0 27312 \f[R] 27313 .fi 27314 .PP 27315 and run the stack: 27316 .IP 27317 .nf 27318 \f[C] 27319 docker stack deploy example -c ./example.yml 27320 \f[R] 27321 .fi 27322 .PP 27323 After a few seconds docker will spread the parsed stack description over 27324 cluster, create the \f[C]example_heimdall\f[R] service on port 27325 \f[I]8080\f[R], run service containers on one or more cluster nodes and 27326 request the \f[C]example_configdata\f[R] volume from rclone plugins on 27327 the node hosts. 27328 You can use the following commands to confirm results: 27329 .IP 27330 .nf 27331 \f[C] 27332 docker service ls 27333 docker service ps example_heimdall 27334 docker volume ls 27335 \f[R] 27336 .fi 27337 .PP 27338 Point your browser to \f[C]http://cluster.host.address:8080\f[R] and 27339 play with the service. 27340 Stop it with \f[C]docker stack remove example\f[R] when you are done. 27341 Note that the \f[C]example_configdata\f[R] volume(s) created on demand 27342 at the cluster nodes will not be automatically removed together with the 27343 stack but stay for future reuse. 27344 You can remove them manually by invoking the 27345 \f[C]docker volume remove example_configdata\f[R] command on every node. 27346 .SS Creating Volumes via CLI 27347 .PP 27348 Volumes can be created with docker volume 27349 create (https://docs.docker.com/engine/reference/commandline/volume_create/). 27350 Here are a few examples: 27351 .IP 27352 .nf 27353 \f[C] 27354 docker volume create vol1 -d rclone -o remote=storj: -o vfs-cache-mode=full 27355 docker volume create vol2 -d rclone -o remote=:storj,access_grant=xxx:heimdall 27356 docker volume create vol3 -d rclone -o type=storj -o path=heimdall -o storj-access-grant=xxx -o poll-interval=0 27357 \f[R] 27358 .fi 27359 .PP 27360 Note the \f[C]-d rclone\f[R] flag that tells docker to request volume 27361 from the rclone driver. 27362 This works even if you installed managed driver by its full name 27363 \f[C]rclone/docker-volume-rclone\f[R] because you provided the 27364 \f[C]--alias rclone\f[R] option. 27365 .PP 27366 Volumes can be inspected as follows: 27367 .IP 27368 .nf 27369 \f[C] 27370 docker volume list 27371 docker volume inspect vol1 27372 \f[R] 27373 .fi 27374 .SS Volume Configuration 27375 .PP 27376 Rclone flags and volume options are set via the \f[C]-o\f[R] flag to the 27377 \f[C]docker volume create\f[R] command. 27378 They include backend-specific parameters as well as mount and 27379 \f[I]VFS\f[R] options. 27380 Also there are a few special \f[C]-o\f[R] options: \f[C]remote\f[R], 27381 \f[C]fs\f[R], \f[C]type\f[R], \f[C]path\f[R], \f[C]mount-type\f[R] and 27382 \f[C]persist\f[R]. 27383 .PP 27384 \f[C]remote\f[R] determines an existing remote name from the config 27385 file, with trailing colon and optionally with a remote path. 27386 See the full syntax in the rclone 27387 documentation (https://rclone.org/docs/#syntax-of-remote-paths). 27388 This option can be aliased as \f[C]fs\f[R] to prevent confusion with the 27389 \f[I]remote\f[R] parameter of such backends as \f[I]crypt\f[R] or 27390 \f[I]alias\f[R]. 27391 .PP 27392 The \f[C]remote=:backend:dir/subdir\f[R] syntax can be used to create 27393 on-the-fly (config-less) 27394 remotes (https://rclone.org/docs/#backend-path-to-dir), while the 27395 \f[C]type\f[R] and \f[C]path\f[R] options provide a simpler alternative 27396 for this. 27397 Using two split options 27398 .IP 27399 .nf 27400 \f[C] 27401 -o type=backend -o path=dir/subdir 27402 \f[R] 27403 .fi 27404 .PP 27405 is equivalent to the combined syntax 27406 .IP 27407 .nf 27408 \f[C] 27409 -o remote=:backend:dir/subdir 27410 \f[R] 27411 .fi 27412 .PP 27413 but is arguably easier to parameterize in scripts. 27414 The \f[C]path\f[R] part is optional. 27415 .PP 27416 Mount and VFS 27417 options (https://rclone.org/commands/rclone_serve_docker/#options) as 27418 well as backend parameters (https://rclone.org/flags/#backend-flags) are 27419 named like their twin command-line flags without the \f[C]--\f[R] CLI 27420 prefix. 27421 Optionally you can use underscores instead of dashes in option names. 27422 For example, \f[C]--vfs-cache-mode full\f[R] becomes 27423 \f[C]-o vfs-cache-mode=full\f[R] or \f[C]-o vfs_cache_mode=full\f[R]. 27424 Boolean CLI flags without value will gain the \f[C]true\f[R] value, e.g. 27425 \f[C]--allow-other\f[R] becomes \f[C]-o allow-other=true\f[R] or 27426 \f[C]-o allow_other=true\f[R]. 27427 .PP 27428 Please note that you can provide parameters only for the backend 27429 immediately referenced by the backend type of mounted \f[C]remote\f[R]. 27430 If this is a wrapping backend like \f[I]alias, chunker or crypt\f[R], 27431 you cannot provide options for the referred to remote or backend. 27432 This limitation is imposed by the rclone connection string parser. 27433 The only workaround is to feed plugin with \f[C]rclone.conf\f[R] or 27434 configure plugin arguments (see below). 27435 .SS Special Volume Options 27436 .PP 27437 \f[C]mount-type\f[R] determines the mount method and in general can be 27438 one of: \f[C]mount\f[R], \f[C]cmount\f[R], or \f[C]mount2\f[R]. 27439 This can be aliased as \f[C]mount_type\f[R]. 27440 It should be noted that the managed rclone docker plugin currently does 27441 not support the \f[C]cmount\f[R] method and \f[C]mount2\f[R] is rarely 27442 needed. 27443 This option defaults to the first found method, which is usually 27444 \f[C]mount\f[R] so you generally won\[aq]t need it. 27445 .PP 27446 \f[C]persist\f[R] is a reserved boolean (true/false) option. 27447 In future it will allow to persist on-the-fly remotes in the plugin 27448 \f[C]rclone.conf\f[R] file. 27449 .SS Connection Strings 27450 .PP 27451 The \f[C]remote\f[R] value can be extended with connection 27452 strings (https://rclone.org/docs/#connection-strings) as an alternative 27453 way to supply backend parameters. 27454 This is equivalent to the \f[C]-o\f[R] backend options with one 27455 \f[I]syntactic difference\f[R]. 27456 Inside connection string the backend prefix must be dropped from 27457 parameter names but in the \f[C]-o param=value\f[R] array it must be 27458 present. 27459 For instance, compare the following option array 27460 .IP 27461 .nf 27462 \f[C] 27463 -o remote=:sftp:/home -o sftp-host=localhost 27464 \f[R] 27465 .fi 27466 .PP 27467 with equivalent connection string: 27468 .IP 27469 .nf 27470 \f[C] 27471 -o remote=:sftp,host=localhost:/home 27472 \f[R] 27473 .fi 27474 .PP 27475 This difference exists because flag options \f[C]-o key=val\f[R] include 27476 not only backend parameters but also mount/VFS flags and possibly other 27477 settings. 27478 Also it allows to discriminate the \f[C]remote\f[R] option from the 27479 \f[C]crypt-remote\f[R] (or similarly named backend parameters) and 27480 arguably simplifies scripting due to clearer value substitution. 27481 .SS Using with Swarm or Compose 27482 .PP 27483 Both \f[I]Docker Swarm\f[R] and \f[I]Docker Compose\f[R] use 27484 YAML (http://yaml.org/spec/1.2/spec.html)-formatted text files to 27485 describe groups (stacks) of containers, their properties, networks and 27486 volumes. 27487 \f[I]Compose\f[R] uses the compose 27488 v2 (https://docs.docker.com/compose/compose-file/compose-file-v2/#volume-configuration-reference) 27489 format, \f[I]Swarm\f[R] uses the compose 27490 v3 (https://docs.docker.com/compose/compose-file/compose-file-v3/#volume-configuration-reference) 27491 format. 27492 They are mostly similar, differences are explained in the docker 27493 documentation (https://docs.docker.com/compose/compose-file/compose-versioning/#upgrading). 27494 .PP 27495 Volumes are described by the children of the top-level 27496 \f[C]volumes:\f[R] node. 27497 Each of them should be named after its volume and have at least two 27498 elements, the self-explanatory \f[C]driver: rclone\f[R] value and the 27499 \f[C]driver_opts:\f[R] structure playing the same role as 27500 \f[C]-o key=val\f[R] CLI flags: 27501 .IP 27502 .nf 27503 \f[C] 27504 volumes: 27505 volume_name_1: 27506 driver: rclone 27507 driver_opts: 27508 remote: \[aq]gdrive:\[aq] 27509 allow_other: \[aq]true\[aq] 27510 vfs_cache_mode: full 27511 token: \[aq]{\[dq]type\[dq]: \[dq]borrower\[dq], \[dq]expires\[dq]: \[dq]2021-12-31\[dq]}\[aq] 27512 poll_interval: 0 27513 \f[R] 27514 .fi 27515 .PP 27516 Notice a few important details: - YAML prefers \f[C]_\f[R] in option 27517 names instead of \f[C]-\f[R]. 27518 - YAML treats single and double quotes interchangeably. 27519 Simple strings and integers can be left unquoted. 27520 - Boolean values must be quoted like \f[C]\[aq]true\[aq]\f[R] or 27521 \f[C]\[dq]false\[dq]\f[R] because these two words are reserved by YAML. 27522 - The filesystem string is keyed with \f[C]remote\f[R] (or with 27523 \f[C]fs\f[R]). 27524 Normally you can omit quotes here, but if the string ends with colon, 27525 you \f[B]must\f[R] quote it like 27526 \f[C]remote: \[dq]storage_box:\[dq]\f[R]. 27527 - YAML is picky about surrounding braces in values as this is in fact 27528 another syntax for key/value 27529 mappings (http://yaml.org/spec/1.2/spec.html#id2790832). 27530 For example, JSON access tokens usually contain double quotes and 27531 surrounding braces, so you must put them in single quotes. 27532 .SS Installing as Managed Plugin 27533 .PP 27534 Docker daemon can install plugins from an image registry and run them 27535 managed. 27536 We maintain the 27537 docker-volume-rclone (https://hub.docker.com/p/rclone/docker-volume-rclone/) 27538 plugin image on Docker Hub (https://hub.docker.com). 27539 .PP 27540 Rclone volume plugin requires \f[B]Docker Engine >= 19.03.15\f[R] 27541 .PP 27542 The plugin requires presence of two directories on the host before it 27543 can be installed. 27544 Note that plugin will \f[B]not\f[R] create them automatically. 27545 By default they must exist on host at the following locations (though 27546 you can tweak the paths): - 27547 \f[C]/var/lib/docker-plugins/rclone/config\f[R] is reserved for the 27548 \f[C]rclone.conf\f[R] config file and \f[B]must\f[R] exist even if 27549 it\[aq]s empty and the config file is not present. 27550 - \f[C]/var/lib/docker-plugins/rclone/cache\f[R] holds the plugin state 27551 file as well as optional VFS caches. 27552 .PP 27553 You can install managed 27554 plugin (https://docs.docker.com/engine/reference/commandline/plugin_install/) 27555 with default settings as follows: 27556 .IP 27557 .nf 27558 \f[C] 27559 docker plugin install rclone/docker-volume-rclone:amd64 --grant-all-permissions --alias rclone 27560 \f[R] 27561 .fi 27562 .PP 27563 The \f[C]:amd64\f[R] part of the image specification after colon is 27564 called a \f[I]tag\f[R]. 27565 Usually you will want to install the latest plugin for your 27566 architecture. 27567 In this case the tag will just name it, like \f[C]amd64\f[R] above. 27568 The following plugin architectures are currently available: - 27569 \f[C]amd64\f[R] - \f[C]arm64\f[R] - \f[C]arm-v7\f[R] 27570 .PP 27571 Sometimes you might want a concrete plugin version, not the latest one. 27572 Then you should use image tag in the form 27573 \f[C]:ARCHITECTURE-VERSION\f[R]. 27574 For example, to install plugin version \f[C]v1.56.2\f[R] on architecture 27575 \f[C]arm64\f[R] you will use tag \f[C]arm64-1.56.2\f[R] (note the 27576 removed \f[C]v\f[R]) so the full image specification becomes 27577 \f[C]rclone/docker-volume-rclone:arm64-1.56.2\f[R]. 27578 .PP 27579 We also provide the \f[C]latest\f[R] plugin tag, but since docker does 27580 not support multi-architecture plugins as of the time of this writing, 27581 this tag is currently an \f[B]alias for \f[CB]amd64\f[B]\f[R]. 27582 By convention the \f[C]latest\f[R] tag is the default one and can be 27583 omitted, thus both \f[C]rclone/docker-volume-rclone:latest\f[R] and just 27584 \f[C]rclone/docker-volume-rclone\f[R] will refer to the latest plugin 27585 release for the \f[C]amd64\f[R] platform. 27586 .PP 27587 Also the \f[C]amd64\f[R] part can be omitted from the versioned rclone 27588 plugin tags. 27589 For example, rclone image reference 27590 \f[C]rclone/docker-volume-rclone:amd64-1.56.2\f[R] can be abbreviated as 27591 \f[C]rclone/docker-volume-rclone:1.56.2\f[R] for convenience. 27592 However, for non-intel architectures you still have to use the full tag 27593 as \f[C]amd64\f[R] or \f[C]latest\f[R] will fail to start. 27594 .PP 27595 Managed plugin is in fact a special container running in a namespace 27596 separate from normal docker containers. 27597 Inside it runs the \f[C]rclone serve docker\f[R] command. 27598 The config and cache directories are bind-mounted into the container at 27599 start. 27600 The docker daemon connects to a unix socket created by the command 27601 inside the container. 27602 The command creates on-demand remote mounts right inside, then docker 27603 machinery propagates them through kernel mount namespaces and 27604 bind-mounts into requesting user containers. 27605 .PP 27606 You can tweak a few plugin settings after installation when it\[aq]s 27607 disabled (not in use), for instance: 27608 .IP 27609 .nf 27610 \f[C] 27611 docker plugin disable rclone 27612 docker plugin set rclone RCLONE_VERBOSE=2 config=/etc/rclone args=\[dq]--vfs-cache-mode=writes --allow-other\[dq] 27613 docker plugin enable rclone 27614 docker plugin inspect rclone 27615 \f[R] 27616 .fi 27617 .PP 27618 Note that if docker refuses to disable the plugin, you should find and 27619 remove all active volumes connected with it as well as containers and 27620 swarm services that use them. 27621 This is rather tedious so please carefully plan in advance. 27622 .PP 27623 You can tweak the following settings: \f[C]args\f[R], \f[C]config\f[R], 27624 \f[C]cache\f[R], \f[C]HTTP_PROXY\f[R], \f[C]HTTPS_PROXY\f[R], 27625 \f[C]NO_PROXY\f[R] and \f[C]RCLONE_VERBOSE\f[R]. 27626 It\[aq]s \f[I]your\f[R] task to keep plugin settings in sync across 27627 swarm cluster nodes. 27628 .PP 27629 \f[C]args\f[R] sets command-line arguments for the 27630 \f[C]rclone serve docker\f[R] command (\f[I]none\f[R] by default). 27631 Arguments should be separated by space so you will normally want to put 27632 them in quotes on the docker plugin 27633 set (https://docs.docker.com/engine/reference/commandline/plugin_set/) 27634 command line. 27635 Both serve docker 27636 flags (https://rclone.org/commands/rclone_serve_docker/#options) and 27637 generic rclone flags (https://rclone.org/flags/) are supported, 27638 including backend parameters that will be used as defaults for volume 27639 creation. 27640 Note that plugin will fail (due to this docker 27641 bug (https://github.com/moby/moby/blob/v20.10.7/plugin/v2/plugin.go#L195)) 27642 if the \f[C]args\f[R] value is empty. 27643 Use e.g. 27644 \f[C]args=\[dq]-v\[dq]\f[R] as a workaround. 27645 .PP 27646 \f[C]config=/host/dir\f[R] sets alternative host location for the config 27647 directory. 27648 Plugin will look for \f[C]rclone.conf\f[R] here. 27649 It\[aq]s not an error if the config file is not present but the 27650 directory must exist. 27651 Please note that plugin can periodically rewrite the config file, for 27652 example when it renews storage access tokens. 27653 Keep this in mind and try to avoid races between the plugin and other 27654 instances of rclone on the host that might try to change the config 27655 simultaneously resulting in corrupted \f[C]rclone.conf\f[R]. 27656 You can also put stuff like private key files for SFTP remotes in this 27657 directory. 27658 Just note that it\[aq]s bind-mounted inside the plugin container at the 27659 predefined path \f[C]/data/config\f[R]. 27660 For example, if your key file is named \f[C]sftp-box1.key\f[R] on the 27661 host, the corresponding volume config option should read 27662 \f[C]-o sftp-key-file=/data/config/sftp-box1.key\f[R]. 27663 .PP 27664 \f[C]cache=/host/dir\f[R] sets alternative host location for the 27665 \f[I]cache\f[R] directory. 27666 The plugin will keep VFS caches here. 27667 Also it will create and maintain the \f[C]docker-plugin.state\f[R] file 27668 in this directory. 27669 When the plugin is restarted or reinstalled, it will look in this file 27670 to recreate any volumes that existed previously. 27671 However, they will not be re-mounted into consuming containers after 27672 restart. 27673 Usually this is not a problem as the docker daemon normally will restart 27674 affected user containers after failures, daemon restarts or host 27675 reboots. 27676 .PP 27677 \f[C]RCLONE_VERBOSE\f[R] sets plugin verbosity from \f[C]0\f[R] (errors 27678 only, by default) to \f[C]2\f[R] (debugging). 27679 Verbosity can be also tweaked via \f[C]args=\[dq]-v [-v] ...\[dq]\f[R]. 27680 Since arguments are more generic, you will rarely need this setting. 27681 The plugin output by default feeds the docker daemon log on local host. 27682 Log entries are reflected as \f[I]errors\f[R] in the docker log but 27683 retain their actual level assigned by rclone in the encapsulated message 27684 string. 27685 .PP 27686 \f[C]HTTP_PROXY\f[R], \f[C]HTTPS_PROXY\f[R], \f[C]NO_PROXY\f[R] 27687 customize the plugin proxy settings. 27688 .PP 27689 You can set custom plugin options right when you install it, \f[I]in one 27690 go\f[R]: 27691 .IP 27692 .nf 27693 \f[C] 27694 docker plugin remove rclone 27695 docker plugin install rclone/docker-volume-rclone:amd64 \[rs] 27696 --alias rclone --grant-all-permissions \[rs] 27697 args=\[dq]-v --allow-other\[dq] config=/etc/rclone 27698 docker plugin inspect rclone 27699 \f[R] 27700 .fi 27701 .SS Healthchecks 27702 .PP 27703 The docker plugin volume protocol doesn\[aq]t provide a way for plugins 27704 to inform the docker daemon that a volume is (un-)available. 27705 As a workaround you can setup a healthcheck to verify that the mount is 27706 responding, for example: 27707 .IP 27708 .nf 27709 \f[C] 27710 services: 27711 my_service: 27712 image: my_image 27713 healthcheck: 27714 test: ls /path/to/rclone/mount || exit 1 27715 interval: 1m 27716 timeout: 15s 27717 retries: 3 27718 start_period: 15s 27719 \f[R] 27720 .fi 27721 .SS Running Plugin under Systemd 27722 .PP 27723 In most cases you should prefer managed mode. 27724 Moreover, MacOS and Windows do not support native Docker plugins. 27725 Please use managed mode on these systems. 27726 Proceed further only if you are on Linux. 27727 .PP 27728 First, install rclone (https://rclone.org/install/). 27729 You can just run it (type \f[C]rclone serve docker\f[R] and hit enter) 27730 for the test. 27731 .PP 27732 Install \f[I]FUSE\f[R]: 27733 .IP 27734 .nf 27735 \f[C] 27736 sudo apt-get -y install fuse 27737 \f[R] 27738 .fi 27739 .PP 27740 Download two systemd configuration files: 27741 docker-volume-rclone.service (https://raw.githubusercontent.com/artpar/artpar/master/contrib/docker-plugin/systemd/docker-volume-rclone.service) 27742 and 27743 docker-volume-rclone.socket (https://raw.githubusercontent.com/artpar/artpar/master/contrib/docker-plugin/systemd/docker-volume-rclone.socket). 27744 .PP 27745 Put them to the \f[C]/etc/systemd/system/\f[R] directory: 27746 .IP 27747 .nf 27748 \f[C] 27749 cp docker-volume-plugin.service /etc/systemd/system/ 27750 cp docker-volume-plugin.socket /etc/systemd/system/ 27751 \f[R] 27752 .fi 27753 .PP 27754 Please note that all commands in this section must be run as 27755 \f[I]root\f[R] but we omit \f[C]sudo\f[R] prefix for brevity. 27756 Now create directories required by the service: 27757 .IP 27758 .nf 27759 \f[C] 27760 mkdir -p /var/lib/docker-volumes/rclone 27761 mkdir -p /var/lib/docker-plugins/rclone/config 27762 mkdir -p /var/lib/docker-plugins/rclone/cache 27763 \f[R] 27764 .fi 27765 .PP 27766 Run the docker plugin service in the socket activated mode: 27767 .IP 27768 .nf 27769 \f[C] 27770 systemctl daemon-reload 27771 systemctl start docker-volume-rclone.service 27772 systemctl enable docker-volume-rclone.socket 27773 systemctl start docker-volume-rclone.socket 27774 systemctl restart docker 27775 \f[R] 27776 .fi 27777 .PP 27778 Or run the service directly: - run \f[C]systemctl daemon-reload\f[R] to 27779 let systemd pick up new config - run 27780 \f[C]systemctl enable docker-volume-rclone.service\f[R] to make the new 27781 service start automatically when you power on your machine. 27782 - run \f[C]systemctl start docker-volume-rclone.service\f[R] to start 27783 the service now. 27784 - run \f[C]systemctl restart docker\f[R] to restart docker daemon and 27785 let it detect the new plugin socket. 27786 Note that this step is not needed in managed mode where docker knows 27787 about plugin state changes. 27788 .PP 27789 The two methods are equivalent from the user perspective, but I 27790 personally prefer socket activation. 27791 .SS Troubleshooting 27792 .PP 27793 You can see managed plugin 27794 settings (https://docs.docker.com/engine/extend/#debugging-plugins) with 27795 .IP 27796 .nf 27797 \f[C] 27798 docker plugin list 27799 docker plugin inspect rclone 27800 \f[R] 27801 .fi 27802 .PP 27803 Note that docker (including latest 20.10.7) will not show actual values 27804 of \f[C]args\f[R], just the defaults. 27805 .PP 27806 Use \f[C]journalctl --unit docker\f[R] to see managed plugin output as 27807 part of the docker daemon log. 27808 Note that docker reflects plugin lines as \f[I]errors\f[R] but their 27809 actual level can be seen from encapsulated message string. 27810 .PP 27811 You will usually install the latest version of managed plugin for your 27812 platform. 27813 Use the following commands to print the actual installed version: 27814 .IP 27815 .nf 27816 \f[C] 27817 PLUGID=$(docker plugin list --no-trunc | awk \[aq]/rclone/{print$1}\[aq]) 27818 sudo runc --root /run/docker/runtime-runc/plugins.moby exec $PLUGID rclone version 27819 \f[R] 27820 .fi 27821 .PP 27822 You can even use \f[C]runc\f[R] to run shell inside the plugin 27823 container: 27824 .IP 27825 .nf 27826 \f[C] 27827 sudo runc --root /run/docker/runtime-runc/plugins.moby exec --tty $PLUGID bash 27828 \f[R] 27829 .fi 27830 .PP 27831 Also you can use curl to check the plugin socket connectivity: 27832 .IP 27833 .nf 27834 \f[C] 27835 docker plugin list --no-trunc 27836 PLUGID=123abc... 27837 sudo curl -H Content-Type:application/json -XPOST -d {} --unix-socket /run/docker/plugins/$PLUGID/rclone.sock http://localhost/Plugin.Activate 27838 \f[R] 27839 .fi 27840 .PP 27841 though this is rarely needed. 27842 .SS Caveats 27843 .PP 27844 Finally I\[aq]d like to mention a \f[I]caveat with updating volume 27845 settings\f[R]. 27846 Docker CLI does not have a dedicated command like 27847 \f[C]docker volume update\f[R]. 27848 It may be tempting to invoke \f[C]docker volume create\f[R] with updated 27849 options on existing volume, but there is a gotcha. 27850 The command will do nothing, it won\[aq]t even return an error. 27851 I hope that docker maintainers will fix this some day. 27852 In the meantime be aware that you must remove your volume before 27853 recreating it with new settings: 27854 .IP 27855 .nf 27856 \f[C] 27857 docker volume remove my_vol 27858 docker volume create my_vol -d rclone -o opt1=new_val1 ... 27859 \f[R] 27860 .fi 27861 .PP 27862 and verify that settings did update: 27863 .IP 27864 .nf 27865 \f[C] 27866 docker volume list 27867 docker volume inspect my_vol 27868 \f[R] 27869 .fi 27870 .PP 27871 If docker refuses to remove the volume, you should find containers or 27872 swarm services that use it and stop them first. 27873 .SS Bisync 27874 .PP 27875 \f[C]bisync\f[R] is \f[B]in beta\f[R] and is considered an \f[B]advanced 27876 command\f[R], so use with care. 27877 Make sure you have read and understood the entire 27878 manual (https://rclone.org/bisync) (especially the Limitations section) 27879 before using, or data loss can result. 27880 Questions can be asked in the Rclone Forum (https://forum.rclone.org/). 27881 .SS Getting started 27882 .IP \[bu] 2 27883 Install rclone (https://rclone.org/install/) and setup your remotes. 27884 .IP \[bu] 2 27885 Bisync will create its working directory at 27886 \f[C]\[ti]/.cache/rclone/bisync\f[R] on Linux, 27887 \f[C]/Users/yourusername/Library/Caches/rclone/bisync\f[R] on Mac, or 27888 \f[C]C:\[rs]Users\[rs]MyLogin\[rs]AppData\[rs]Local\[rs]rclone\[rs]bisync\f[R] 27889 on Windows. 27890 Make sure that this location is writable. 27891 .IP \[bu] 2 27892 Run bisync with the \f[C]--resync\f[R] flag, specifying the paths to the 27893 local and remote sync directory roots. 27894 .IP \[bu] 2 27895 For successive sync runs, leave off the \f[C]--resync\f[R] flag. 27896 (\f[B]Important!\f[R]) 27897 .IP \[bu] 2 27898 Consider using a filters file for excluding unnecessary files and 27899 directories from the sync. 27900 .IP \[bu] 2 27901 Consider setting up the --check-access feature for safety. 27902 .IP \[bu] 2 27903 On Linux or Mac, consider setting up a crontab entry. 27904 bisync can safely run in concurrent cron jobs thanks to lock files it 27905 maintains. 27906 .PP 27907 For example, your first command might look like this: 27908 .IP 27909 .nf 27910 \f[C] 27911 rclone bisync remote1:path1 remote2:path2 --create-empty-src-dirs --compare size,modtime,checksum --slow-hash-sync-only --resilient -MvP --drive-skip-gdocs --fix-case --resync --dry-run 27912 \f[R] 27913 .fi 27914 .PP 27915 If all looks good, run it again without \f[C]--dry-run\f[R]. 27916 After that, remove \f[C]--resync\f[R] as well. 27917 .PP 27918 Here is a typical run log (with timestamps removed for clarity): 27919 .IP 27920 .nf 27921 \f[C] 27922 rclone bisync /testdir/path1/ /testdir/path2/ --verbose 27923 INFO : Synching Path1 \[dq]/testdir/path1/\[dq] with Path2 \[dq]/testdir/path2/\[dq] 27924 INFO : Path1 checking for diffs 27925 INFO : - Path1 File is new - file11.txt 27926 INFO : - Path1 File is newer - file2.txt 27927 INFO : - Path1 File is newer - file5.txt 27928 INFO : - Path1 File is newer - file7.txt 27929 INFO : - Path1 File was deleted - file4.txt 27930 INFO : - Path1 File was deleted - file6.txt 27931 INFO : - Path1 File was deleted - file8.txt 27932 INFO : Path1: 7 changes: 1 new, 3 newer, 0 older, 3 deleted 27933 INFO : Path2 checking for diffs 27934 INFO : - Path2 File is new - file10.txt 27935 INFO : - Path2 File is newer - file1.txt 27936 INFO : - Path2 File is newer - file5.txt 27937 INFO : - Path2 File is newer - file6.txt 27938 INFO : - Path2 File was deleted - file3.txt 27939 INFO : - Path2 File was deleted - file7.txt 27940 INFO : - Path2 File was deleted - file8.txt 27941 INFO : Path2: 7 changes: 1 new, 3 newer, 0 older, 3 deleted 27942 INFO : Applying changes 27943 INFO : - Path1 Queue copy to Path2 - /testdir/path2/file11.txt 27944 INFO : - Path1 Queue copy to Path2 - /testdir/path2/file2.txt 27945 INFO : - Path2 Queue delete - /testdir/path2/file4.txt 27946 NOTICE: - WARNING New or changed in both paths - file5.txt 27947 NOTICE: - Path1 Renaming Path1 copy - /testdir/path1/file5.txt..path1 27948 NOTICE: - Path1 Queue copy to Path2 - /testdir/path2/file5.txt..path1 27949 NOTICE: - Path2 Renaming Path2 copy - /testdir/path2/file5.txt..path2 27950 NOTICE: - Path2 Queue copy to Path1 - /testdir/path1/file5.txt..path2 27951 INFO : - Path2 Queue copy to Path1 - /testdir/path1/file6.txt 27952 INFO : - Path1 Queue copy to Path2 - /testdir/path2/file7.txt 27953 INFO : - Path2 Queue copy to Path1 - /testdir/path1/file1.txt 27954 INFO : - Path2 Queue copy to Path1 - /testdir/path1/file10.txt 27955 INFO : - Path1 Queue delete - /testdir/path1/file3.txt 27956 INFO : - Path2 Do queued copies to - Path1 27957 INFO : - Path1 Do queued copies to - Path2 27958 INFO : - Do queued deletes on - Path1 27959 INFO : - Do queued deletes on - Path2 27960 INFO : Updating listings 27961 INFO : Validating listings for Path1 \[dq]/testdir/path1/\[dq] vs Path2 \[dq]/testdir/path2/\[dq] 27962 INFO : Bisync successful 27963 \f[R] 27964 .fi 27965 .SS Command line syntax 27966 .IP 27967 .nf 27968 \f[C] 27969 $ rclone bisync --help 27970 Usage: 27971 rclone bisync remote1:path1 remote2:path2 [flags] 27972 27973 Positional arguments: 27974 Path1, Path2 Local path, or remote storage with \[aq]:\[aq] plus optional path. 27975 Type \[aq]rclone listremotes\[aq] for list of configured remotes. 27976 27977 Optional Flags: 27978 --backup-dir1 string --backup-dir for Path1. Must be a non-overlapping path on the same remote. 27979 --backup-dir2 string --backup-dir for Path2. Must be a non-overlapping path on the same remote. 27980 --check-access Ensure expected RCLONE_TEST files are found on both Path1 and Path2 filesystems, else abort. 27981 --check-filename string Filename for --check-access (default: RCLONE_TEST) 27982 --check-sync string Controls comparison of final listings: true|false|only (default: true) (default \[dq]true\[dq]) 27983 --compare string Comma-separated list of bisync-specific compare options ex. \[aq]size,modtime,checksum\[aq] (default: \[aq]size,modtime\[aq]) 27984 --conflict-loser ConflictLoserAction Action to take on the loser of a sync conflict (when there is a winner) or on both files (when there is no winner): , num, pathname, delete (default: num) 27985 --conflict-resolve string Automatically resolve conflicts by preferring the version that is: none, path1, path2, newer, older, larger, smaller (default: none) (default \[dq]none\[dq]) 27986 --conflict-suffix string Suffix to use when renaming a --conflict-loser. Can be either one string or two comma-separated strings to assign different suffixes to Path1/Path2. (default: \[aq]conflict\[aq]) 27987 --create-empty-src-dirs Sync creation and deletion of empty directories. (Not compatible with --remove-empty-dirs) 27988 --download-hash Compute hash by downloading when otherwise unavailable. (warning: may be slow and use lots of data!) 27989 --filters-file string Read filtering patterns from a file 27990 --force Bypass --max-delete safety check and run the sync. Consider using with --verbose 27991 -h, --help help for bisync 27992 --ignore-listing-checksum Do not use checksums for listings (add --ignore-checksum to additionally skip post-copy checksum checks) 27993 --max-lock Duration Consider lock files older than this to be expired (default: 0 (never expire)) (minimum: 2m) (default 0s) 27994 --no-cleanup Retain working files (useful for troubleshooting and testing). 27995 --no-slow-hash Ignore listing checksums only on backends where they are slow 27996 --recover Automatically recover from interruptions without requiring --resync. 27997 --remove-empty-dirs Remove ALL empty directories at the final cleanup step. 27998 --resilient Allow future runs to retry after certain less-serious errors, instead of requiring --resync. Use at your own risk! 27999 -1, --resync Performs the resync run. Equivalent to --resync-mode path1. Consider using --verbose or --dry-run first. 28000 --resync-mode string During resync, prefer the version that is: path1, path2, newer, older, larger, smaller (default: path1 if --resync, otherwise none for no resync.) (default \[dq]none\[dq]) 28001 --retries int Retry operations this many times if they fail (requires --resilient). (default 3) 28002 --retries-sleep Duration Interval between retrying operations if they fail, e.g. 500ms, 60s, 5m (0 to disable) (default 0s) 28003 --slow-hash-sync-only Ignore slow checksums for listings and deltas, but still consider them during sync calls. 28004 --workdir string Use custom working dir - useful for testing. (default: {WORKDIR}) 28005 --max-delete PERCENT Safety check on maximum percentage of deleted files allowed. If exceeded, the bisync run will abort. (default: 50%) 28006 -n, --dry-run Go through the motions - No files are copied/deleted. 28007 -v, --verbose Increases logging verbosity. May be specified more than once for more details. 28008 \f[R] 28009 .fi 28010 .PP 28011 Arbitrary rclone flags may be specified on the bisync command 28012 line (https://rclone.org/commands/rclone_bisync/), for example 28013 \f[C]rclone bisync ./testdir/path1/ gdrive:testdir/path2/ --drive-skip-gdocs -v -v --timeout 10s\f[R] 28014 Note that interactions of various rclone flags with bisync process flow 28015 has not been fully tested yet. 28016 .SS Paths 28017 .PP 28018 Path1 and Path2 arguments may be references to any mix of local 28019 directory paths (absolute or relative), UNC paths 28020 (\f[C]//server/share/path\f[R]), Windows drive paths (with a drive 28021 letter and \f[C]:\f[R]) or configured 28022 remotes (https://rclone.org/docs/#syntax-of-remote-paths) with optional 28023 subdirectory paths. 28024 Cloud references are distinguished by having a \f[C]:\f[R] in the 28025 argument (see Windows support below). 28026 .PP 28027 Path1 and Path2 are treated equally, in that neither has priority for 28028 file changes (except during \f[C]--resync\f[R]), and access efficiency 28029 does not change whether a remote is on Path1 or Path2. 28030 .PP 28031 The listings in bisync working directory (default: 28032 \f[C]\[ti]/.cache/rclone/bisync\f[R]) are named based on the Path1 and 28033 Path2 arguments so that separate syncs to individual directories within 28034 the tree may be set up, e.g.: 28035 \f[C]path_to_local_tree..dropbox_subdir.lst\f[R]. 28036 .PP 28037 Any empty directories after the sync on both the Path1 and Path2 28038 filesystems are not deleted by default, unless 28039 \f[C]--create-empty-src-dirs\f[R] is specified. 28040 If the \f[C]--remove-empty-dirs\f[R] flag is specified, then both paths 28041 will have ALL empty directories purged as the last step in the process. 28042 .SS Command-line flags 28043 .SS --resync 28044 .PP 28045 This will effectively make both Path1 and Path2 filesystems contain a 28046 matching superset of all files. 28047 By default, Path2 files that do not exist in Path1 will be copied to 28048 Path1, and the process will then copy the Path1 tree to Path2. 28049 .PP 28050 The \f[C]--resync\f[R] sequence is roughly equivalent to the following 28051 (but see \f[C]--resync-mode\f[R] for other options): 28052 .IP 28053 .nf 28054 \f[C] 28055 rclone copy Path2 Path1 --ignore-existing [--create-empty-src-dirs] 28056 rclone copy Path1 Path2 [--create-empty-src-dirs] 28057 \f[R] 28058 .fi 28059 .PP 28060 The base directories on both Path1 and Path2 filesystems must exist or 28061 bisync will fail. 28062 This is required for safety - that bisync can verify that both paths are 28063 valid. 28064 .PP 28065 When using \f[C]--resync\f[R], a newer version of a file on the Path2 28066 filesystem will (by default) be overwritten by the Path1 filesystem 28067 version. 28068 (Note that this is NOT entirely 28069 symmetrical (https://github.com/artpar/artpar/issues/5681#issuecomment-938761815), 28070 and more symmetrical options can be specified with the 28071 \f[C]--resync-mode\f[R] flag.) Carefully evaluate deltas using 28072 --dry-run (https://rclone.org/flags/#non-backend-flags). 28073 .PP 28074 For a resync run, one of the paths may be empty (no files in the path 28075 tree). 28076 The resync run should result in files on both paths, else a normal 28077 non-resync run will fail. 28078 .PP 28079 For a non-resync run, either path being empty (no files in the tree) 28080 fails with 28081 \f[C]Empty current PathN listing. Cannot sync to an empty directory: X.pathN.lst\f[R] 28082 This is a safety check that an unexpected empty path does not result in 28083 deleting \f[B]everything\f[R] in the other path. 28084 .PP 28085 Note that \f[C]--resync\f[R] implies \f[C]--resync-mode path1\f[R] 28086 unless a different \f[C]--resync-mode\f[R] is explicitly specified. 28087 It is not necessary to use both the \f[C]--resync\f[R] and 28088 \f[C]--resync-mode\f[R] flags -- either one is sufficient without the 28089 other. 28090 .PP 28091 \f[B]Note:\f[R] \f[C]--resync\f[R] (including \f[C]--resync-mode\f[R]) 28092 should only be used under three specific (rare) circumstances: 1. 28093 It is your \f[I]first\f[R] bisync run (between these two paths) 2. 28094 You\[aq]ve just made changes to your bisync settings (such as editing 28095 the contents of your \f[C]--filters-file\f[R]) 3. 28096 There was an error on the prior run, and as a result, bisync now 28097 requires \f[C]--resync\f[R] to recover 28098 .PP 28099 The rest of the time, you should \f[I]omit\f[R] \f[C]--resync\f[R]. 28100 The reason is because \f[C]--resync\f[R] will only \f[I]copy\f[R] (not 28101 \f[I]sync\f[R]) each side to the other. 28102 Therefore, if you included \f[C]--resync\f[R] for every bisync run, it 28103 would never be possible to delete a file -- the deleted file would 28104 always keep reappearing at the end of every run (because it\[aq]s being 28105 copied from the other side where it still exists). 28106 Similarly, renaming a file would always result in a duplicate copy (both 28107 old and new name) on both sides. 28108 .PP 28109 If you find that frequent interruptions from #3 are an issue, rather 28110 than automatically running \f[C]--resync\f[R], the recommended 28111 alternative is to use the \f[C]--resilient\f[R], \f[C]--recover\f[R], 28112 and \f[C]--conflict-resolve\f[R] flags, (along with Graceful Shutdown 28113 mode, when needed) for a very robust \[dq]set-it-and-forget-it\[dq] 28114 bisync setup that can automatically bounce back from almost any 28115 interruption it might encounter. 28116 Consider adding something like the following: 28117 .IP 28118 .nf 28119 \f[C] 28120 --resilient --recover --max-lock 2m --conflict-resolve newer 28121 \f[R] 28122 .fi 28123 .SS --resync-mode CHOICE 28124 .PP 28125 In the event that a file differs on both sides during a 28126 \f[C]--resync\f[R], \f[C]--resync-mode\f[R] controls which version will 28127 overwrite the other. 28128 The supported options are similar to \f[C]--conflict-resolve\f[R]. 28129 For all of the following options, the version that is kept is referred 28130 to as the \[dq]winner\[dq], and the version that is overwritten 28131 (deleted) is referred to as the \[dq]loser\[dq]. 28132 The options are named after the \[dq]winner\[dq]: 28133 .IP \[bu] 2 28134 \f[C]path1\f[R] - (the default) - the version from Path1 is 28135 unconditionally considered the winner (regardless of \f[C]modtime\f[R] 28136 and \f[C]size\f[R], if any). 28137 This can be useful if one side is more trusted or up-to-date than the 28138 other, at the time of the \f[C]--resync\f[R]. 28139 .IP \[bu] 2 28140 \f[C]path2\f[R] - same as \f[C]path1\f[R], except the path2 version is 28141 considered the winner. 28142 .IP \[bu] 2 28143 \f[C]newer\f[R] - the newer file (by \f[C]modtime\f[R]) is considered 28144 the winner, regardless of which side it came from. 28145 This may result in having a mix of some winners from Path1, and some 28146 winners from Path2. 28147 (The implementation is analogous to running 28148 \f[C]rclone copy --update\f[R] in both directions.) 28149 .IP \[bu] 2 28150 \f[C]older\f[R] - same as \f[C]newer\f[R], except the older file is 28151 considered the winner, and the newer file is considered the loser. 28152 .IP \[bu] 2 28153 \f[C]larger\f[R] - the larger file (by \f[C]size\f[R]) is considered the 28154 winner (regardless of \f[C]modtime\f[R], if any). 28155 This can be a useful option for remotes without \f[C]modtime\f[R] 28156 support, or with the kinds of files (such as logs) that tend to grow but 28157 not shrink, over time. 28158 .IP \[bu] 2 28159 \f[C]smaller\f[R] - the smaller file (by \f[C]size\f[R]) is considered 28160 the winner (regardless of \f[C]modtime\f[R], if any). 28161 .PP 28162 For all of the above options, note the following: - If either of the 28163 underlying remotes lacks support for the chosen method, it will be 28164 ignored and will fall back to the default of \f[C]path1\f[R]. 28165 (For example, if \f[C]--resync-mode newer\f[R] is set, but one of the 28166 paths uses a remote that doesn\[aq]t support \f[C]modtime\f[R].) - If a 28167 winner can\[aq]t be determined because the chosen method\[aq]s attribute 28168 is missing or equal, it will be ignored, and bisync will instead try to 28169 determine whether the files differ by looking at the other 28170 \f[C]--compare\f[R] methods in effect. 28171 (For example, if \f[C]--resync-mode newer\f[R] is set, but the Path1 and 28172 Path2 modtimes are identical, bisync will compare the sizes.) If bisync 28173 concludes that they differ, preference is given to whichever is the 28174 \[dq]source\[dq] at that moment. 28175 (In practice, this gives a slight advantage to Path2, as the 2to1 copy 28176 comes before the 1to2 copy.) If the files \f[I]do not\f[R] differ, 28177 nothing is copied (as both sides are already correct). 28178 - These options apply only to files that exist on both sides (with the 28179 same name and relative path). 28180 Files that exist \f[I]only\f[R] on one side and not the other are 28181 \f[I]always\f[R] copied to the other, during \f[C]--resync\f[R] (this is 28182 one of the main differences between resync and non-resync runs.). 28183 - \f[C]--conflict-resolve\f[R], \f[C]--conflict-loser\f[R], and 28184 \f[C]--conflict-suffix\f[R] do not apply during \f[C]--resync\f[R], and 28185 unlike these flags, nothing is renamed during \f[C]--resync\f[R]. 28186 When a file differs on both sides during \f[C]--resync\f[R], one version 28187 always overwrites the other (much like in \f[C]rclone copy\f[R].) 28188 (Consider using \f[C]--backup-dir\f[R] to retain a backup of the losing 28189 version.) - Unlike for \f[C]--conflict-resolve\f[R], 28190 \f[C]--resync-mode none\f[R] is not a valid option (or rather, it will 28191 be interpreted as \[dq]no resync\[dq], unless \f[C]--resync\f[R] has 28192 also been specified, in which case it will be ignored.) - Winners and 28193 losers are decided at the individual file-level only (there is not 28194 currently an option to pick an entire winning directory atomically, 28195 although the \f[C]path1\f[R] and \f[C]path2\f[R] options typically 28196 produce a similar result.) - To maintain backward-compatibility, the 28197 \f[C]--resync\f[R] flag implies \f[C]--resync-mode path1\f[R] unless a 28198 different \f[C]--resync-mode\f[R] is explicitly specified. 28199 Similarly, all \f[C]--resync-mode\f[R] options (except \f[C]none\f[R]) 28200 imply \f[C]--resync\f[R], so it is not necessary to use both the 28201 \f[C]--resync\f[R] and \f[C]--resync-mode\f[R] flags simultaneously -- 28202 either one is sufficient without the other. 28203 .SS --check-access 28204 .PP 28205 Access check files are an additional safety measure against data loss. 28206 bisync will ensure it can find matching \f[C]RCLONE_TEST\f[R] files in 28207 the same places in the Path1 and Path2 filesystems. 28208 \f[C]RCLONE_TEST\f[R] files are not generated automatically. 28209 For \f[C]--check-access\f[R] to succeed, you must first either: 28210 \f[B]A)\f[R] Place one or more \f[C]RCLONE_TEST\f[R] files in both 28211 systems, or \f[B]B)\f[R] Set \f[C]--check-filename\f[R] to a filename 28212 already in use in various locations throughout your sync\[aq]d fileset. 28213 Recommended methods for \f[B]A)\f[R] include: * 28214 \f[C]rclone touch Path1/RCLONE_TEST\f[R] (create a new file) * 28215 \f[C]rclone copyto Path1/RCLONE_TEST Path2/RCLONE_TEST\f[R] (copy an 28216 existing file) * 28217 \f[C]rclone copy Path1/RCLONE_TEST Path2/RCLONE_TEST --include \[dq]RCLONE_TEST\[dq]\f[R] 28218 (copy multiple files at once, recursively) * create the files manually 28219 (outside of rclone) * run \f[C]bisync\f[R] once \f[I]without\f[R] 28220 \f[C]--check-access\f[R] to set matching files on both filesystems will 28221 also work, but is not preferred, due to potential for user error (you 28222 are temporarily disabling the safety feature). 28223 .PP 28224 Note that \f[C]--check-access\f[R] is still enforced on 28225 \f[C]--resync\f[R], so \f[C]bisync --resync --check-access\f[R] will not 28226 work as a method of initially setting the files (this is to ensure that 28227 bisync can\[aq]t inadvertently circumvent its own safety 28228 switch (https://forum.rclone.org/t/bisync-bugs-and-feature-requests/37636#:~:text=3.%20%2D%2Dcheck%2Daccess%20doesn%27t%20always%20fail%20when%20it%20should).) 28229 .PP 28230 Time stamps and file contents for \f[C]RCLONE_TEST\f[R] files are not 28231 important, just the names and locations. 28232 If you have symbolic links in your sync tree it is recommended to place 28233 \f[C]RCLONE_TEST\f[R] files in the linked-to directory tree to protect 28234 against bisync assuming a bunch of deleted files if the linked-to tree 28235 should not be accessible. 28236 See also the --check-filename flag. 28237 .SS --check-filename 28238 .PP 28239 Name of the file(s) used in access health validation. 28240 The default \f[C]--check-filename\f[R] is \f[C]RCLONE_TEST\f[R]. 28241 One or more files having this filename must exist, synchronized between 28242 your source and destination filesets, in order for 28243 \f[C]--check-access\f[R] to succeed. 28244 See --check-access for additional details. 28245 .SS --compare 28246 .PP 28247 As of \f[C]v1.66\f[R], bisync fully supports comparing based on any 28248 combination of size, modtime, and checksum (lifting the prior 28249 restriction on backends without modtime support.) 28250 .PP 28251 By default (without the \f[C]--compare\f[R] flag), bisync inherits the 28252 same comparison options as \f[C]sync\f[R] (that is: \f[C]size\f[R] and 28253 \f[C]modtime\f[R] by default, unless modified with flags such as 28254 \f[C]--checksum\f[R] (https://rclone.org/docs/#c-checksum) or 28255 \f[C]--size-only\f[R].) 28256 .PP 28257 If the \f[C]--compare\f[R] flag is set, it will override these defaults. 28258 This can be useful if you wish to compare based on combinations not 28259 currently supported in \f[C]sync\f[R], such as comparing all three of 28260 \f[C]size\f[R] AND \f[C]modtime\f[R] AND \f[C]checksum\f[R] 28261 simultaneously (or just \f[C]modtime\f[R] AND \f[C]checksum\f[R]). 28262 .PP 28263 \f[C]--compare\f[R] takes a comma-separated list, with the currently 28264 supported values being \f[C]size\f[R], \f[C]modtime\f[R], and 28265 \f[C]checksum\f[R]. 28266 For example, if you want to compare size and checksum, but not modtime, 28267 you would do: 28268 .IP 28269 .nf 28270 \f[C] 28271 --compare size,checksum 28272 \f[R] 28273 .fi 28274 .PP 28275 Or if you want to compare all three: 28276 .IP 28277 .nf 28278 \f[C] 28279 --compare size,modtime,checksum 28280 \f[R] 28281 .fi 28282 .PP 28283 \f[C]--compare\f[R] overrides any conflicting flags. 28284 For example, if you set the conflicting flags 28285 \f[C]--compare checksum --size-only\f[R], \f[C]--size-only\f[R] will be 28286 ignored, and bisync will compare checksum and not size. 28287 To avoid confusion, it is recommended to use \f[I]either\f[R] 28288 \f[C]--compare\f[R] or the normal \f[C]sync\f[R] flags, but not both. 28289 .PP 28290 If \f[C]--compare\f[R] includes \f[C]checksum\f[R] and both remotes 28291 support checksums but have no hash types in common with each other, 28292 checksums will be considered \f[I]only\f[R] for comparisons within the 28293 same side (to determine what has changed since the prior sync), but not 28294 for comparisons against the opposite side. 28295 If one side supports checksums and the other does not, checksums will 28296 only be considered on the side that supports them. 28297 .PP 28298 When comparing with \f[C]checksum\f[R] and/or \f[C]size\f[R] without 28299 \f[C]modtime\f[R], bisync cannot determine whether a file is 28300 \f[C]newer\f[R] or \f[C]older\f[R] -- only whether it is 28301 \f[C]changed\f[R] or \f[C]unchanged\f[R]. 28302 (If it is \f[C]changed\f[R] on both sides, bisync still does the 28303 standard equality-check to avoid declaring a sync conflict unless it 28304 absolutely has to.) 28305 .PP 28306 It is recommended to do a \f[C]--resync\f[R] when changing 28307 \f[C]--compare\f[R] settings, as otherwise your prior listing files may 28308 not contain the attributes you wish to compare (for example, they will 28309 not have stored checksums if you were not previously comparing 28310 checksums.) 28311 .SS --ignore-listing-checksum 28312 .PP 28313 When \f[C]--checksum\f[R] or \f[C]--compare checksum\f[R] is set, bisync 28314 will retrieve (or generate) checksums (for backends that support them) 28315 when creating the listings for both paths, and store the checksums in 28316 the listing files. 28317 \f[C]--ignore-listing-checksum\f[R] will disable this behavior, which 28318 may speed things up considerably, especially on backends (such as 28319 local (https://rclone.org/local/)) where hashes must be computed on the 28320 fly instead of retrieved. 28321 Please note the following: 28322 .IP \[bu] 2 28323 As of \f[C]v1.66\f[R], \f[C]--ignore-listing-checksum\f[R] is now 28324 automatically set when neither \f[C]--checksum\f[R] nor 28325 \f[C]--compare checksum\f[R] are in use (as the checksums would not be 28326 used for anything.) 28327 .IP \[bu] 2 28328 \f[C]--ignore-listing-checksum\f[R] is NOT the same as 28329 \f[C]--ignore-checksum\f[R] (https://rclone.org/docs/#ignore-checksum), 28330 and you may wish to use one or the other, or both. 28331 In a nutshell: \f[C]--ignore-listing-checksum\f[R] controls whether 28332 checksums are considered when scanning for diffs, while 28333 \f[C]--ignore-checksum\f[R] controls whether checksums are considered 28334 during the copy/sync operations that follow, if there ARE diffs. 28335 .IP \[bu] 2 28336 Unless \f[C]--ignore-listing-checksum\f[R] is passed, bisync currently 28337 computes hashes for one path \f[I]even when there\[aq]s no common hash 28338 with the other path\f[R] (for example, a 28339 crypt (https://rclone.org/crypt/#modification-times-and-hashes) remote.) 28340 This can still be beneficial, as the hashes will still be used to detect 28341 changes within the same side (if \f[C]--checksum\f[R] or 28342 \f[C]--compare checksum\f[R] is set), even if they can\[aq]t be used to 28343 compare against the opposite side. 28344 .IP \[bu] 2 28345 If you wish to ignore listing checksums \f[I]only\f[R] on remotes where 28346 they are slow to compute, consider using \f[C]--no-slow-hash\f[R] (or 28347 \f[C]--slow-hash-sync-only\f[R]) instead of 28348 \f[C]--ignore-listing-checksum\f[R]. 28349 .IP \[bu] 2 28350 If \f[C]--ignore-listing-checksum\f[R] is used simultaneously with 28351 \f[C]--compare checksum\f[R] (or \f[C]--checksum\f[R]), checksums will 28352 be ignored for bisync deltas, but still considered during the sync 28353 operations that follow (if deltas are detected based on modtime and/or 28354 size.) 28355 .SS --no-slow-hash 28356 .PP 28357 On some remotes (notably \f[C]local\f[R]), checksums can dramatically 28358 slow down a bisync run, because hashes cannot be stored and need to be 28359 computed in real-time when they are requested. 28360 On other remotes (such as \f[C]drive\f[R]), they add practically no time 28361 at all. 28362 The \f[C]--no-slow-hash\f[R] flag will automatically skip checksums on 28363 remotes where they are slow, while still comparing them on others 28364 (assuming \f[C]--compare\f[R] includes \f[C]checksum\f[R].) This can be 28365 useful when one of your bisync paths is slow but you still want to check 28366 checksums on the other, for a more robust sync. 28367 .SS --slow-hash-sync-only 28368 .PP 28369 Same as \f[C]--no-slow-hash\f[R], except slow hashes are still 28370 considered during sync calls. 28371 They are still NOT considered for determining deltas, nor or they 28372 included in listings. 28373 They are also skipped during \f[C]--resync\f[R]. 28374 The main use case for this flag is when you have a large number of 28375 files, but relatively few of them change from run to run -- so you 28376 don\[aq]t want to check your entire tree every time (it would take too 28377 long), but you still want to consider checksums for the smaller group of 28378 files for which a \f[C]modtime\f[R] or \f[C]size\f[R] change was 28379 detected. 28380 Keep in mind that this speed savings comes with a safety trade-off: if a 28381 file\[aq]s content were to change without a change to its 28382 \f[C]modtime\f[R] or \f[C]size\f[R], bisync would not detect it, and it 28383 would not be synced. 28384 .PP 28385 \f[C]--slow-hash-sync-only\f[R] is only useful if both remotes share a 28386 common hash type (if they don\[aq]t, bisync will automatically fall back 28387 to \f[C]--no-slow-hash\f[R].) Both \f[C]--no-slow-hash\f[R] and 28388 \f[C]--slow-hash-sync-only\f[R] have no effect without 28389 \f[C]--compare checksum\f[R] (or \f[C]--checksum\f[R]). 28390 .SS --download-hash 28391 .PP 28392 If \f[C]--download-hash\f[R] is set, bisync will use best efforts to 28393 obtain an MD5 checksum by downloading and computing on-the-fly, when 28394 checksums are not otherwise available (for example, a remote that 28395 doesn\[aq]t support them.) Note that since rclone has to download the 28396 entire file, this may dramatically slow down your bisync runs, and is 28397 also likely to use a lot of data, so it is probably not practical for 28398 bisync paths with a large total file size. 28399 However, it can be a good option for syncing small-but-important files 28400 with maximum accuracy (for example, a source code repo on a 28401 \f[C]crypt\f[R] remote.) An additional advantage over methods like 28402 \f[C]cryptcheck\f[R] (https://rclone.org/commands/rclone_cryptcheck/) is 28403 that the original file is not required for comparison (for example, 28404 \f[C]--download-hash\f[R] can be used to bisync two different crypt 28405 remotes with different passwords.) 28406 .PP 28407 When \f[C]--download-hash\f[R] is set, bisync still looks for more 28408 efficient checksums first, and falls back to downloading only when none 28409 are found. 28410 It takes priority over conflicting flags such as 28411 \f[C]--no-slow-hash\f[R]. 28412 \f[C]--download-hash\f[R] is not suitable for Google Docs and other 28413 files of unknown size, as their checksums would change from run to run 28414 (due to small variances in the internals of the generated export file.) 28415 Therefore, bisync automatically skips \f[C]--download-hash\f[R] for 28416 files with a size less than 0. 28417 .PP 28418 See also: \f[C]Hasher\f[R] (https://rclone.org/hasher/) backend, 28419 \f[C]cryptcheck\f[R] (https://rclone.org/commands/rclone_cryptcheck/) 28420 command, 28421 \f[C]rclone check --download\f[R] (https://rclone.org/commands/rclone_check/) 28422 option, \f[C]md5sum\f[R] (https://rclone.org/commands/rclone_md5sum/) 28423 command 28424 .SS --max-delete 28425 .PP 28426 As a safety check, if greater than the \f[C]--max-delete\f[R] percent of 28427 files were deleted on either the Path1 or Path2 filesystem, then bisync 28428 will abort with a warning message, without making any changes. 28429 The default \f[C]--max-delete\f[R] is \f[C]50%\f[R]. 28430 One way to trigger this limit is to rename a directory that contains 28431 more than half of your files. 28432 This will appear to bisync as a bunch of deleted files and a bunch of 28433 new files. 28434 This safety check is intended to block bisync from deleting all of the 28435 files on both filesystems due to a temporary network access issue, or if 28436 the user had inadvertently deleted the files on one side or the other. 28437 To force the sync, either set a different delete percentage limit, e.g. 28438 \f[C]--max-delete 75\f[R] (allows up to 75% deletion), or use 28439 \f[C]--force\f[R] to bypass the check. 28440 .PP 28441 Also see the all files changed check. 28442 .SS --filters-file 28443 .PP 28444 By using rclone filter features you can exclude file types or directory 28445 sub-trees from the sync. 28446 See the bisync filters section and generic 28447 --filter-from (https://rclone.org/filtering/#filter-from-read-filtering-patterns-from-a-file) 28448 documentation. 28449 An example filters file contains filters for non-allowed files for 28450 synching with Dropbox. 28451 .PP 28452 If you make changes to your filters file then bisync requires a run with 28453 \f[C]--resync\f[R]. 28454 This is a safety feature, which prevents existing files on the Path1 28455 and/or Path2 side from seeming to disappear from view (since they are 28456 excluded in the new listings), which would fool bisync into seeing them 28457 as deleted (as compared to the prior run listings), and then bisync 28458 would proceed to delete them for real. 28459 .PP 28460 To block this from happening, bisync calculates an MD5 hash of the 28461 filters file and stores the hash in a \f[C].md5\f[R] file in the same 28462 place as your filters file. 28463 On the next run with \f[C]--filters-file\f[R] set, bisync re-calculates 28464 the MD5 hash of the current filters file and compares it to the hash 28465 stored in the \f[C].md5\f[R] file. 28466 If they don\[aq]t match, the run aborts with a critical error and thus 28467 forces you to do a \f[C]--resync\f[R], likely avoiding a disaster. 28468 .SS --conflict-resolve CHOICE 28469 .PP 28470 In bisync, a \[dq]conflict\[dq] is a file that is \f[I]new\f[R] or 28471 \f[I]changed\f[R] on \f[I]both sides\f[R] (relative to the prior run) 28472 AND is \f[I]not currently identical\f[R] on both sides. 28473 \f[C]--conflict-resolve\f[R] controls how bisync handles such a 28474 scenario. 28475 The currently supported options are: 28476 .IP \[bu] 2 28477 \f[C]none\f[R] - (the default) - do not attempt to pick a winner, keep 28478 and rename both files according to \f[C]--conflict-loser\f[R] and 28479 \f[C]--conflict-suffix\f[R] settings. 28480 For example, with the default settings, \f[C]file.txt\f[R] on Path1 is 28481 renamed \f[C]file.txt.conflict1\f[R] and \f[C]file.txt\f[R] on Path2 is 28482 renamed \f[C]file.txt.conflict2\f[R]. 28483 Both are copied to the opposite path during the run, so both sides end 28484 up with a copy of both files. 28485 (As \f[C]none\f[R] is the default, it is not necessary to specify 28486 \f[C]--conflict-resolve none\f[R] -- you can just omit the flag.) 28487 .IP \[bu] 2 28488 \f[C]newer\f[R] - the newer file (by \f[C]modtime\f[R]) is considered 28489 the winner and is copied without renaming. 28490 The older file (the \[dq]loser\[dq]) is handled according to 28491 \f[C]--conflict-loser\f[R] and \f[C]--conflict-suffix\f[R] settings 28492 (either renamed or deleted.) For example, if \f[C]file.txt\f[R] on Path1 28493 is newer than \f[C]file.txt\f[R] on Path2, the result on both sides 28494 (with other default settings) will be \f[C]file.txt\f[R] (winner from 28495 Path1) and \f[C]file.txt.conflict1\f[R] (loser from Path2). 28496 .IP \[bu] 2 28497 \f[C]older\f[R] - same as \f[C]newer\f[R], except the older file is 28498 considered the winner, and the newer file is considered the loser. 28499 .IP \[bu] 2 28500 \f[C]larger\f[R] - the larger file (by \f[C]size\f[R]) is considered the 28501 winner (regardless of \f[C]modtime\f[R], if any). 28502 .IP \[bu] 2 28503 \f[C]smaller\f[R] - the smaller file (by \f[C]size\f[R]) is considered 28504 the winner (regardless of \f[C]modtime\f[R], if any). 28505 .IP \[bu] 2 28506 \f[C]path1\f[R] - the version from Path1 is unconditionally considered 28507 the winner (regardless of \f[C]modtime\f[R] and \f[C]size\f[R], if any). 28508 This can be useful if one side is usually more trusted or up-to-date 28509 than the other. 28510 .IP \[bu] 2 28511 \f[C]path2\f[R] - same as \f[C]path1\f[R], except the path2 version is 28512 considered the winner. 28513 .PP 28514 For all of the above options, note the following: - If either of the 28515 underlying remotes lacks support for the chosen method, it will be 28516 ignored and fall back to \f[C]none\f[R]. 28517 (For example, if \f[C]--conflict-resolve newer\f[R] is set, but one of 28518 the paths uses a remote that doesn\[aq]t support \f[C]modtime\f[R].) - 28519 If a winner can\[aq]t be determined because the chosen method\[aq]s 28520 attribute is missing or equal, it will be ignored and fall back to 28521 \f[C]none\f[R]. 28522 (For example, if \f[C]--conflict-resolve newer\f[R] is set, but the 28523 Path1 and Path2 modtimes are identical, even if the sizes may differ.) - 28524 If the file\[aq]s content is currently identical on both sides, it is 28525 not considered a \[dq]conflict\[dq], even if new or changed on both 28526 sides since the prior sync. 28527 (For example, if you made a change on one side and then synced it to the 28528 other side by other means.) Therefore, none of the conflict resolution 28529 flags apply in this scenario. 28530 - The conflict resolution flags do not apply during a 28531 \f[C]--resync\f[R], as there is no \[dq]prior run\[dq] to speak of (but 28532 see \f[C]--resync-mode\f[R] for similar options.) 28533 .SS --conflict-loser CHOICE 28534 .PP 28535 \f[C]--conflict-loser\f[R] determines what happens to the 28536 \[dq]loser\[dq] of a sync conflict (when \f[C]--conflict-resolve\f[R] 28537 determines a winner) or to both files (when there is no winner.) The 28538 currently supported options are: 28539 .IP \[bu] 2 28540 \f[C]num\f[R] - (the default) - auto-number the conflicts by 28541 automatically appending the next available number to the 28542 \f[C]--conflict-suffix\f[R], in chronological order. 28543 For example, with the default settings, the first conflict for 28544 \f[C]file.txt\f[R] will be renamed \f[C]file.txt.conflict1\f[R]. 28545 If \f[C]file.txt.conflict1\f[R] already exists, 28546 \f[C]file.txt.conflict2\f[R] will be used instead (etc., up to a maximum 28547 of 9223372036854775807 conflicts.) 28548 .IP \[bu] 2 28549 \f[C]pathname\f[R] - rename the conflicts according to which side they 28550 came from, which was the default behavior prior to \f[C]v1.66\f[R]. 28551 For example, with \f[C]--conflict-suffix path\f[R], \f[C]file.txt\f[R] 28552 from Path1 will be renamed \f[C]file.txt.path1\f[R], and 28553 \f[C]file.txt\f[R] from Path2 will be renamed \f[C]file.txt.path2\f[R]. 28554 If two non-identical suffixes are provided (ex. 28555 \f[C]--conflict-suffix cloud,local\f[R]), the trailing digit is omitted. 28556 Importantly, note that with \f[C]pathname\f[R], there is no 28557 auto-numbering beyond \f[C]2\f[R], so if \f[C]file.txt.path2\f[R] 28558 somehow already exists, it will be overwritten. 28559 Using a dynamic date variable in your \f[C]--conflict-suffix\f[R] (see 28560 below) is one possible way to avoid this. 28561 Note also that conflicts-of-conflicts are possible, if the original 28562 conflict is not manually resolved -- for example, if for some reason you 28563 edited \f[C]file.txt.path1\f[R] on both sides, and those edits were 28564 different, the result would be \f[C]file.txt.path1.path1\f[R] and 28565 \f[C]file.txt.path1.path2\f[R] (in addition to 28566 \f[C]file.txt.path2\f[R].) 28567 .IP \[bu] 2 28568 \f[C]delete\f[R] - keep the winner only and delete the loser, instead of 28569 renaming it. 28570 If a winner cannot be determined (see \f[C]--conflict-resolve\f[R] for 28571 details on how this could happen), \f[C]delete\f[R] is ignored and the 28572 default \f[C]num\f[R] is used instead (i.e. 28573 both versions are kept and renamed, and neither is deleted.) 28574 \f[C]delete\f[R] is inherently the most destructive option, so use it 28575 only with care. 28576 .PP 28577 For all of the above options, note that if a winner cannot be determined 28578 (see \f[C]--conflict-resolve\f[R] for details on how this could happen), 28579 or if \f[C]--conflict-resolve\f[R] is not in use, \f[I]both\f[R] files 28580 will be renamed. 28581 .SS --conflict-suffix STRING[,STRING] 28582 .PP 28583 \f[C]--conflict-suffix\f[R] controls the suffix that is appended when 28584 bisync renames a \f[C]--conflict-loser\f[R] (default: 28585 \f[C]conflict\f[R]). 28586 \f[C]--conflict-suffix\f[R] will accept either one string or two 28587 comma-separated strings to assign different suffixes to Path1 vs. 28588 Path2. 28589 This may be helpful later in identifying the source of the conflict. 28590 (For example, 28591 \f[C]--conflict-suffix dropboxconflict,laptopconflict\f[R]) 28592 .PP 28593 With \f[C]--conflict-loser num\f[R], a number is always appended to the 28594 suffix. 28595 With \f[C]--conflict-loser pathname\f[R], a number is appended only when 28596 one suffix is specified (or when two identical suffixes are specified.) 28597 i.e. 28598 with \f[C]--conflict-loser pathname\f[R], all of the following would 28599 produce exactly the same result: 28600 .IP 28601 .nf 28602 \f[C] 28603 --conflict-suffix path 28604 --conflict-suffix path,path 28605 --conflict-suffix path1,path2 28606 \f[R] 28607 .fi 28608 .PP 28609 Suffixes may be as short as 1 character. 28610 By default, the suffix is appended after any other extensions (ex. 28611 \f[C]file.jpg.conflict1\f[R]), however, this can be changed with the 28612 \f[C]--suffix-keep-extension\f[R] (https://rclone.org/docs/#suffix-keep-extension) 28613 flag (i.e. 28614 to instead result in \f[C]file.conflict1.jpg\f[R]). 28615 .PP 28616 \f[C]--conflict-suffix\f[R] supports several \f[I]dynamic date 28617 variables\f[R] when enclosed in curly braces as globs. 28618 This can be helpful to track the date and/or time that each conflict was 28619 handled by bisync. 28620 For example: 28621 .IP 28622 .nf 28623 \f[C] 28624 --conflict-suffix {DateOnly}-conflict 28625 // result: myfile.txt.2006-01-02-conflict1 28626 \f[R] 28627 .fi 28628 .PP 28629 All of the formats described 28630 here (https://pkg.go.dev/time#pkg-constants) and 28631 here (https://pkg.go.dev/time#example-Time.Format) are supported, but 28632 take care to ensure that your chosen format does not use any characters 28633 that are illegal on your remotes (for example, macOS does not allow 28634 colons in filenames, and slashes are also best avoided as they are often 28635 interpreted as directory separators.) To address this particular issue, 28636 an additional \f[C]{MacFriendlyTime}\f[R] (or just \f[C]{mac}\f[R]) 28637 option is supported, which results in \f[C]2006-01-02 0304PM\f[R]. 28638 .PP 28639 Note that \f[C]--conflict-suffix\f[R] is entirely separate from 28640 rclone\[aq]s main 28641 \f[C]--sufix\f[R] (https://rclone.org/docs/#suffix-suffix) flag. 28642 This is intentional, as users may wish to use both flags simultaneously, 28643 if also using \f[C]--backup-dir\f[R]. 28644 .PP 28645 Finally, note that the default in bisync prior to \f[C]v1.66\f[R] was to 28646 rename conflicts with \f[C]..path1\f[R] and \f[C]..path2\f[R] (with two 28647 periods, and \f[C]path\f[R] instead of \f[C]conflict\f[R].) Bisync now 28648 defaults to a single dot instead of a double dot, but additional dots 28649 can be added by including them in the specified suffix string. 28650 For example, for behavior equivalent to the previous default, use: 28651 .IP 28652 .nf 28653 \f[C] 28654 [--conflict-resolve none] --conflict-loser pathname --conflict-suffix .path 28655 \f[R] 28656 .fi 28657 .SS --check-sync 28658 .PP 28659 Enabled by default, the check-sync function checks that all of the same 28660 files exist in both the Path1 and Path2 history listings. 28661 This \f[I]check-sync\f[R] integrity check is performed at the end of the 28662 sync run by default. 28663 Any untrapped failing copy/deletes between the two paths might result in 28664 differences between the two listings and in the untracked file content 28665 differences between the two paths. 28666 A resync run would correct the error. 28667 .PP 28668 Note that the default-enabled integrity check locally executes a load of 28669 both the final Path1 and Path2 listings, and thus adds to the run time 28670 of a sync. 28671 Using \f[C]--check-sync=false\f[R] will disable it and may significantly 28672 reduce the sync run times for very large numbers of files. 28673 .PP 28674 The check may be run manually with \f[C]--check-sync=only\f[R]. 28675 It runs only the integrity check and terminates without actually 28676 synching. 28677 .PP 28678 Note that currently, \f[C]--check-sync\f[R] \f[B]only checks listing 28679 snapshots and NOT the actual files on the remotes.\f[R] Note also that 28680 the listing snapshots will not know about any changes that happened 28681 during or after the latest bisync run, as those will be discovered on 28682 the next run. 28683 Therefore, while listings should always match \f[I]each other\f[R] at 28684 the end of a bisync run, it is \f[I]expected\f[R] that they will not 28685 match the underlying remotes, nor will the remotes match each other, if 28686 there were changes during or after the run. 28687 This is normal, and any differences will be detected and synced on the 28688 next run. 28689 .PP 28690 For a robust integrity check of the current state of the remotes (as 28691 opposed to just their listing snapshots), consider using \f[C]check\f[R] 28692 (or 28693 \f[C]cryptcheck\f[R] (https://rclone.org/commands/rclone_cryptcheck/), 28694 if at least one path is a \f[C]crypt\f[R] remote) instead of 28695 \f[C]--check-sync\f[R], keeping in mind that differences are expected if 28696 files changed during or after your last bisync run. 28697 .PP 28698 For example, a possible sequence could look like this: 28699 .IP "1." 3 28700 Normally scheduled bisync run: 28701 .IP 28702 .nf 28703 \f[C] 28704 rclone bisync Path1 Path2 -MPc --check-access --max-delete 10 --filters-file /path/to/filters.txt -v --no-cleanup --ignore-listing-checksum --disable ListR --checkers=16 --drive-pacer-min-sleep=10ms --create-empty-src-dirs --resilient 28705 \f[R] 28706 .fi 28707 .IP "2." 3 28708 Periodic independent integrity check (perhaps scheduled nightly or 28709 weekly): 28710 .IP 28711 .nf 28712 \f[C] 28713 rclone check -MvPc Path1 Path2 --filter-from /path/to/filters.txt 28714 \f[R] 28715 .fi 28716 .IP "3." 3 28717 If diffs are found, you have some choices to correct them. 28718 If one side is more up-to-date and you want to make the other side match 28719 it, you could run: 28720 .IP 28721 .nf 28722 \f[C] 28723 rclone sync Path1 Path2 --filter-from /path/to/filters.txt --create-empty-src-dirs -MPc -v 28724 \f[R] 28725 .fi 28726 .PP 28727 (or switch Path1 and Path2 to make Path2 the source-of-truth) 28728 .PP 28729 Or, if neither side is totally up-to-date, you could run a 28730 \f[C]--resync\f[R] to bring them back into agreement (but remember that 28731 this could cause deleted files to re-appear.) 28732 .PP 28733 *Note also that \f[C]rclone check\f[R] does not currently include empty 28734 directories, so if you want to know if any empty directories are out of 28735 sync, consider alternatively running the above \f[C]rclone sync\f[R] 28736 command with \f[C]--dry-run\f[R] added. 28737 .PP 28738 See also: Concurrent modifications, \f[C]--resilient\f[R] 28739 .SS --resilient 28740 .PP 28741 \f[B]\f[BI]Caution: this is an experimental feature. Use at your own 28742 risk!\f[B]\f[R] 28743 .PP 28744 By default, most errors or interruptions will cause bisync to abort and 28745 require \f[C]--resync\f[R] to recover. 28746 This is a safety feature, to prevent bisync from running again until a 28747 user checks things out. 28748 However, in some cases, bisync can go too far and enforce a lockout when 28749 one isn\[aq]t actually necessary, like for certain less-serious errors 28750 that might resolve themselves on the next run. 28751 When \f[C]--resilient\f[R] is specified, bisync tries its best to 28752 recover and self-correct, and only requires \f[C]--resync\f[R] as a last 28753 resort when a human\[aq]s involvement is absolutely necessary. 28754 The intended use case is for running bisync as a background process 28755 (such as via scheduled cron). 28756 .PP 28757 When using \f[C]--resilient\f[R] mode, bisync will still report the 28758 error and abort, however it will not lock out future runs -- allowing 28759 the possibility of retrying at the next normally scheduled time, without 28760 requiring a \f[C]--resync\f[R] first. 28761 Examples of such retryable errors include access test failures, missing 28762 listing files, and filter change detections. 28763 These safety features will still prevent the \f[I]current\f[R] run from 28764 proceeding -- the difference is that if conditions have improved by the 28765 time of the \f[I]next\f[R] run, that next run will be allowed to 28766 proceed. 28767 Certain more serious errors will still enforce a \f[C]--resync\f[R] 28768 lockout, even in \f[C]--resilient\f[R] mode, to prevent data loss. 28769 .PP 28770 Behavior of \f[C]--resilient\f[R] may change in a future version. 28771 (See also: \f[C]--recover\f[R], \f[C]--max-lock\f[R], Graceful Shutdown) 28772 .SS --recover 28773 .PP 28774 If \f[C]--recover\f[R] is set, in the event of a sudden interruption or 28775 other un-graceful shutdown, bisync will attempt to automatically recover 28776 on the next run, instead of requiring \f[C]--resync\f[R]. 28777 Bisync is able to recover robustly by keeping one \[dq]backup\[dq] 28778 listing at all times, representing the state of both paths after the 28779 last known successful sync. 28780 Bisync can then compare the current state with this snapshot to 28781 determine which changes it needs to retry. 28782 Changes that were synced after this snapshot (during the run that was 28783 later interrupted) will appear to bisync as if they are \[dq]new or 28784 changed on both sides\[dq], but in most cases this is not a problem, as 28785 bisync will simply do its usual \[dq]equality check\[dq] and learn that 28786 no action needs to be taken on these files, since they are already 28787 identical on both sides. 28788 .PP 28789 In the rare event that a file is synced successfully during a run that 28790 later aborts, and then that same file changes AGAIN before the next run, 28791 bisync will think it is a sync conflict, and handle it accordingly. 28792 (From bisync\[aq]s perspective, the file has changed on both sides since 28793 the last trusted sync, and the files on either side are not currently 28794 identical.) Therefore, \f[C]--recover\f[R] carries with it a slightly 28795 increased chance of having conflicts -- though in practice this is 28796 pretty rare, as the conditions required to cause it are quite specific. 28797 This risk can be reduced by using bisync\[aq]s \[dq]Graceful 28798 Shutdown\[dq] mode (triggered by sending \f[C]SIGINT\f[R] or 28799 \f[C]Ctrl+C\f[R]), when you have the choice, instead of forcing a sudden 28800 termination. 28801 .PP 28802 \f[C]--recover\f[R] and \f[C]--resilient\f[R] are similar, but distinct 28803 -- the main difference is that \f[C]--resilient\f[R] is about 28804 \f[I]retrying\f[R], while \f[C]--recover\f[R] is about 28805 \f[I]recovering\f[R]. 28806 Most users will probably want both. 28807 \f[C]--resilient\f[R] allows retrying when bisync has chosen to abort 28808 itself due to safety features such as failing \f[C]--check-access\f[R] 28809 or detecting a filter change. 28810 \f[C]--resilient\f[R] does not cover external interruptions such as a 28811 user shutting down their computer in the middle of a sync -- that is 28812 what \f[C]--recover\f[R] is for. 28813 .SS --max-lock 28814 .PP 28815 Bisync uses lock files as a safety feature to prevent interference from 28816 other bisync runs while it is running. 28817 Bisync normally removes these lock files at the end of a run, but if 28818 bisync is abruptly interrupted, these files will be left behind. 28819 By default, they will lock out all future runs, until the user has a 28820 chance to manually check things out and remove the lock. 28821 As an alternative, \f[C]--max-lock\f[R] can be used to make them 28822 automatically expire after a certain period of time, so that future runs 28823 are not locked out forever, and auto-recovery is possible. 28824 \f[C]--max-lock\f[R] can be any duration \f[C]2m\f[R] or greater (or 28825 \f[C]0\f[R] to disable). 28826 If set, lock files older than this will be considered \[dq]expired\[dq], 28827 and future runs will be allowed to disregard them and proceed. 28828 (Note that the \f[C]--max-lock\f[R] duration must be set by the process 28829 that left the lock file -- not the later one interpreting it.) 28830 .PP 28831 If set, bisync will also \[dq]renew\[dq] these lock files every 28832 \f[C]--max-lock minus one minute\f[R] throughout a run, for extra 28833 safety. 28834 (For example, with \f[C]--max-lock 5m\f[R], bisync would renew the lock 28835 file (for another 5 minutes) every 4 minutes until the run has 28836 completed.) In other words, it should not be possible for a lock file to 28837 pass its expiration time while the process that created it is still 28838 running -- and you can therefore be reasonably sure that any 28839 \f[I]expired\f[R] lock file you may find was left there by an 28840 interrupted run, not one that is still running and just taking awhile. 28841 .PP 28842 If \f[C]--max-lock\f[R] is \f[C]0\f[R] or not set, the default is that 28843 lock files will never expire, and will block future runs (of these same 28844 two bisync paths) indefinitely. 28845 .PP 28846 For maximum resilience from disruptions, consider setting a relatively 28847 short duration like \f[C]--max-lock 2m\f[R] along with 28848 \f[C]--resilient\f[R] and \f[C]--recover\f[R], and a relatively frequent 28849 cron schedule. 28850 The result will be a very robust \[dq]set-it-and-forget-it\[dq] bisync 28851 run that can automatically bounce back from almost any interruption it 28852 might encounter, without requiring the user to get involved and run a 28853 \f[C]--resync\f[R]. 28854 (See also: Graceful Shutdown mode) 28855 .SS --backup-dir1 and --backup-dir2 28856 .PP 28857 As of \f[C]v1.66\f[R], 28858 \f[C]--backup-dir\f[R] (https://rclone.org/docs/#backup-dir-dir) is 28859 supported in bisync. 28860 Because \f[C]--backup-dir\f[R] must be a non-overlapping path on the 28861 same remote, Bisync has introduced new \f[C]--backup-dir1\f[R] and 28862 \f[C]--backup-dir2\f[R] flags to support separate backup-dirs for 28863 \f[C]Path1\f[R] and \f[C]Path2\f[R] (bisyncing between different remotes 28864 with \f[C]--backup-dir\f[R] would not otherwise be possible.) 28865 \f[C]--backup-dir1\f[R] and \f[C]--backup-dir2\f[R] can use different 28866 remotes from each other, but \f[C]--backup-dir1\f[R] must use the same 28867 remote as \f[C]Path1\f[R], and \f[C]--backup-dir2\f[R] must use the same 28868 remote as \f[C]Path2\f[R]. 28869 Each backup directory must not overlap its respective bisync Path 28870 without being excluded by a filter rule. 28871 .PP 28872 The standard \f[C]--backup-dir\f[R] will also work, if both paths use 28873 the same remote (but note that deleted files from both paths would be 28874 mixed together in the same dir). 28875 If either \f[C]--backup-dir1\f[R] and \f[C]--backup-dir2\f[R] are set, 28876 they will override \f[C]--backup-dir\f[R]. 28877 .PP 28878 Example: 28879 .IP 28880 .nf 28881 \f[C] 28882 rclone bisync /Users/someuser/some/local/path/Bisync gdrive:Bisync --backup-dir1 /Users/someuser/some/local/path/BackupDir --backup-dir2 gdrive:BackupDir --suffix -2023-08-26 --suffix-keep-extension --check-access --max-delete 10 --filters-file /Users/someuser/some/local/path/bisync_filters.txt --no-cleanup --ignore-listing-checksum --checkers=16 --drive-pacer-min-sleep=10ms --create-empty-src-dirs --resilient -MvP --drive-skip-gdocs --fix-case 28883 \f[R] 28884 .fi 28885 .PP 28886 In this example, if the user deletes a file in 28887 \f[C]/Users/someuser/some/local/path/Bisync\f[R], bisync will propagate 28888 the delete to the other side by moving the corresponding file from 28889 \f[C]gdrive:Bisync\f[R] to \f[C]gdrive:BackupDir\f[R]. 28890 If the user deletes a file from \f[C]gdrive:Bisync\f[R], bisync moves it 28891 from \f[C]/Users/someuser/some/local/path/Bisync\f[R] to 28892 \f[C]/Users/someuser/some/local/path/BackupDir\f[R]. 28893 .PP 28894 In the event of a rename due to a sync conflict, the rename is not 28895 considered a delete, unless a previous conflict with the same name 28896 already exists and would get overwritten. 28897 .PP 28898 See also: \f[C]--suffix\f[R] (https://rclone.org/docs/#suffix-suffix), 28899 \f[C]--suffix-keep-extension\f[R] (https://rclone.org/docs/#suffix-keep-extension) 28900 .SS Operation 28901 .SS Runtime flow details 28902 .PP 28903 bisync retains the listings of the \f[C]Path1\f[R] and \f[C]Path2\f[R] 28904 filesystems from the prior run. 28905 On each successive run it will: 28906 .IP \[bu] 2 28907 list files on \f[C]path1\f[R] and \f[C]path2\f[R], and check for changes 28908 on each side. 28909 Changes include \f[C]New\f[R], \f[C]Newer\f[R], \f[C]Older\f[R], and 28910 \f[C]Deleted\f[R] files. 28911 .IP \[bu] 2 28912 Propagate changes on \f[C]path1\f[R] to \f[C]path2\f[R], and vice-versa. 28913 .SS Safety measures 28914 .IP \[bu] 2 28915 Lock file prevents multiple simultaneous runs when taking a while. 28916 This can be particularly useful if bisync is run by cron scheduler. 28917 .IP \[bu] 2 28918 Handle change conflicts non-destructively by creating 28919 \f[C].conflict1\f[R], \f[C].conflict2\f[R], etc. 28920 file versions, according to \f[C]--conflict-resolve\f[R], 28921 \f[C]--conflict-loser\f[R], and \f[C]--conflict-suffix\f[R] settings. 28922 .IP \[bu] 2 28923 File system access health check using \f[C]RCLONE_TEST\f[R] files (see 28924 the \f[C]--check-access\f[R] flag). 28925 .IP \[bu] 2 28926 Abort on excessive deletes - protects against a failed listing being 28927 interpreted as all the files were deleted. 28928 See the \f[C]--max-delete\f[R] and \f[C]--force\f[R] flags. 28929 .IP \[bu] 2 28930 If something evil happens, bisync goes into a safe state to block damage 28931 by later runs. 28932 (See Error Handling) 28933 .SS Normal sync checks 28934 .PP 28935 .TS 28936 tab(@); 28937 lw(8.4n) lw(28.4n) lw(15.7n) lw(17.5n). 28938 T{ 28939 Type 28940 T}@T{ 28941 Description 28942 T}@T{ 28943 Result 28944 T}@T{ 28945 Implementation 28946 T} 28947 _ 28948 T{ 28949 Path2 new 28950 T}@T{ 28951 File is new on Path2, does not exist on Path1 28952 T}@T{ 28953 Path2 version survives 28954 T}@T{ 28955 \f[C]rclone copy\f[R] Path2 to Path1 28956 T} 28957 T{ 28958 Path2 newer 28959 T}@T{ 28960 File is newer on Path2, unchanged on Path1 28961 T}@T{ 28962 Path2 version survives 28963 T}@T{ 28964 \f[C]rclone copy\f[R] Path2 to Path1 28965 T} 28966 T{ 28967 Path2 deleted 28968 T}@T{ 28969 File is deleted on Path2, unchanged on Path1 28970 T}@T{ 28971 File is deleted 28972 T}@T{ 28973 \f[C]rclone delete\f[R] Path1 28974 T} 28975 T{ 28976 Path1 new 28977 T}@T{ 28978 File is new on Path1, does not exist on Path2 28979 T}@T{ 28980 Path1 version survives 28981 T}@T{ 28982 \f[C]rclone copy\f[R] Path1 to Path2 28983 T} 28984 T{ 28985 Path1 newer 28986 T}@T{ 28987 File is newer on Path1, unchanged on Path2 28988 T}@T{ 28989 Path1 version survives 28990 T}@T{ 28991 \f[C]rclone copy\f[R] Path1 to Path2 28992 T} 28993 T{ 28994 Path1 older 28995 T}@T{ 28996 File is older on Path1, unchanged on Path2 28997 T}@T{ 28998 \f[I]Path1 version survives\f[R] 28999 T}@T{ 29000 \f[C]rclone copy\f[R] Path1 to Path2 29001 T} 29002 T{ 29003 Path2 older 29004 T}@T{ 29005 File is older on Path2, unchanged on Path1 29006 T}@T{ 29007 \f[I]Path2 version survives\f[R] 29008 T}@T{ 29009 \f[C]rclone copy\f[R] Path2 to Path1 29010 T} 29011 T{ 29012 Path1 deleted 29013 T}@T{ 29014 File no longer exists on Path1 29015 T}@T{ 29016 File is deleted 29017 T}@T{ 29018 \f[C]rclone delete\f[R] Path2 29019 T} 29020 .TE 29021 .SS Unusual sync checks 29022 .PP 29023 .TS 29024 tab(@); 29025 lw(17.2n) lw(21.0n) lw(19.4n) lw(12.4n). 29026 T{ 29027 Type 29028 T}@T{ 29029 Description 29030 T}@T{ 29031 Result 29032 T}@T{ 29033 Implementation 29034 T} 29035 _ 29036 T{ 29037 Path1 new/changed AND Path2 new/changed AND Path1 == Path2 29038 T}@T{ 29039 File is new/changed on Path1 AND new/changed on Path2 AND Path1 version 29040 is currently identical to Path2 29041 T}@T{ 29042 No change 29043 T}@T{ 29044 None 29045 T} 29046 T{ 29047 Path1 new AND Path2 new 29048 T}@T{ 29049 File is new on Path1 AND new on Path2 (and Path1 version is NOT 29050 identical to Path2) 29051 T}@T{ 29052 Conflicts handled according to \f[C]--conflict-resolve\f[R] & 29053 \f[C]--conflict-loser\f[R] settings 29054 T}@T{ 29055 default: \f[C]rclone copy\f[R] renamed \f[C]Path2.conflict2\f[R] file to 29056 Path1, \f[C]rclone copy\f[R] renamed \f[C]Path1.conflict1\f[R] file to 29057 Path2 29058 T} 29059 T{ 29060 Path2 newer AND Path1 changed 29061 T}@T{ 29062 File is newer on Path2 AND also changed (newer/older/size) on Path1 (and 29063 Path1 version is NOT identical to Path2) 29064 T}@T{ 29065 Conflicts handled according to \f[C]--conflict-resolve\f[R] & 29066 \f[C]--conflict-loser\f[R] settings 29067 T}@T{ 29068 default: \f[C]rclone copy\f[R] renamed \f[C]Path2.conflict2\f[R] file to 29069 Path1, \f[C]rclone copy\f[R] renamed \f[C]Path1.conflict1\f[R] file to 29070 Path2 29071 T} 29072 T{ 29073 Path2 newer AND Path1 deleted 29074 T}@T{ 29075 File is newer on Path2 AND also deleted on Path1 29076 T}@T{ 29077 Path2 version survives 29078 T}@T{ 29079 \f[C]rclone copy\f[R] Path2 to Path1 29080 T} 29081 T{ 29082 Path2 deleted AND Path1 changed 29083 T}@T{ 29084 File is deleted on Path2 AND changed (newer/older/size) on Path1 29085 T}@T{ 29086 Path1 version survives 29087 T}@T{ 29088 \f[C]rclone copy\f[R] Path1 to Path2 29089 T} 29090 T{ 29091 Path1 deleted AND Path2 changed 29092 T}@T{ 29093 File is deleted on Path1 AND changed (newer/older/size) on Path2 29094 T}@T{ 29095 Path2 version survives 29096 T}@T{ 29097 \f[C]rclone copy\f[R] Path2 to Path1 29098 T} 29099 .TE 29100 .PP 29101 As of \f[C]rclone v1.64\f[R], bisync is now better at detecting 29102 \f[I]false positive\f[R] sync conflicts, which would previously have 29103 resulted in unnecessary renames and duplicates. 29104 Now, when bisync comes to a file that it wants to rename (because it is 29105 new/changed on both sides), it first checks whether the Path1 and Path2 29106 versions are currently \f[I]identical\f[R] (using the same underlying 29107 function as \f[C]check\f[R].) If bisync concludes that the files are 29108 identical, it will skip them and move on. 29109 Otherwise, it will create renamed duplicates, as before. 29110 This behavior also improves the experience of renaming 29111 directories (https://forum.rclone.org/t/bisync-bugs-and-feature-requests/37636#:~:text=Renamed%20directories), 29112 as a \f[C]--resync\f[R] is no longer required, so long as the same 29113 change has been made on both sides. 29114 .SS All files changed check 29115 .PP 29116 If \f[I]all\f[R] prior existing files on either of the filesystems have 29117 changed (e.g. 29118 timestamps have changed due to changing the system\[aq]s timezone) then 29119 bisync will abort without making any changes. 29120 Any new files are not considered for this check. 29121 You could use \f[C]--force\f[R] to force the sync (whichever side has 29122 the changed timestamp files wins). 29123 Alternately, a \f[C]--resync\f[R] may be used (Path1 versions will be 29124 pushed to Path2). 29125 Consider the situation carefully and perhaps use \f[C]--dry-run\f[R] 29126 before you commit to the changes. 29127 .SS Modification times 29128 .PP 29129 By default, bisync compares files by modification time and size. 29130 If you or your application should change the content of a file without 29131 changing the modification time and size, then bisync will \f[I]not\f[R] 29132 notice the change, and thus will not copy it to the other side. 29133 As an alternative, consider comparing by checksum (if your remotes 29134 support it). 29135 See \f[C]--compare\f[R] for details. 29136 .SS Error handling 29137 .PP 29138 Certain bisync critical errors, such as file copy/move failing, will 29139 result in a bisync lockout of following runs. 29140 The lockout is asserted because the sync status and history of the Path1 29141 and Path2 filesystems cannot be trusted, so it is safer to block any 29142 further changes until someone checks things out. 29143 The recovery is to do a \f[C]--resync\f[R] again. 29144 .PP 29145 It is recommended to use \f[C]--resync --dry-run --verbose\f[R] 29146 initially and \f[I]carefully\f[R] review what changes will be made 29147 before running the \f[C]--resync\f[R] without \f[C]--dry-run\f[R]. 29148 .PP 29149 Most of these events come up due to an error status from an internal 29150 call. 29151 On such a critical error the \f[C]{...}.path1.lst\f[R] and 29152 \f[C]{...}.path2.lst\f[R] listing files are renamed to extension 29153 \f[C].lst-err\f[R], which blocks any future bisync runs (since the 29154 normal \f[C].lst\f[R] files are not found). 29155 Bisync keeps them under \f[C]bisync\f[R] subdirectory of the rclone 29156 cache directory, typically at \f[C]${HOME}/.cache/rclone/bisync/\f[R] on 29157 Linux. 29158 .PP 29159 Some errors are considered temporary and re-running the bisync is not 29160 blocked. 29161 The \f[I]critical return\f[R] blocks further bisync runs. 29162 .PP 29163 See also: \f[C]--resilient\f[R], \f[C]--recover\f[R], 29164 \f[C]--max-lock\f[R], Graceful Shutdown 29165 .SS Lock file 29166 .PP 29167 When bisync is running, a lock file is created in the bisync working 29168 directory, typically at 29169 \f[C]\[ti]/.cache/rclone/bisync/PATH1..PATH2.lck\f[R] on Linux. 29170 If bisync should crash or hang, the lock file will remain in place and 29171 block any further runs of bisync \f[I]for the same paths\f[R]. 29172 Delete the lock file as part of debugging the situation. 29173 The lock file effectively blocks follow-on (e.g., scheduled by 29174 \f[I]cron\f[R]) runs when the prior invocation is taking a long time. 29175 The lock file contains \f[I]PID\f[R] of the blocking process, which may 29176 help in debug. 29177 Lock files can be set to automatically expire after a certain amount of 29178 time, using the \f[C]--max-lock\f[R] flag. 29179 .PP 29180 \f[B]Note\f[R] that while concurrent bisync runs are allowed, \f[I]be 29181 very cautious\f[R] that there is no overlap in the trees being synched 29182 between concurrent runs, lest there be replicated files, deleted files 29183 and general mayhem. 29184 .SS Return codes 29185 .PP 29186 \f[C]rclone bisync\f[R] returns the following codes to calling program: 29187 - \f[C]0\f[R] on a successful run, - \f[C]1\f[R] for a non-critical 29188 failing run (a rerun may be successful), - \f[C]2\f[R] for a critically 29189 aborted run (requires a \f[C]--resync\f[R] to recover). 29190 .SS Graceful Shutdown 29191 .PP 29192 Bisync has a \[dq]Graceful Shutdown\[dq] mode which is activated by 29193 sending \f[C]SIGINT\f[R] or pressing \f[C]Ctrl+C\f[R] during a run. 29194 Once triggered, bisync will use best efforts to exit cleanly before the 29195 timer runs out. 29196 If bisync is in the middle of transferring files, it will attempt to 29197 cleanly empty its queue by finishing what it has started but not taking 29198 more. 29199 If it cannot do so within 30 seconds, it will cancel the in-progress 29200 transfers at that point and then give itself a maximum of 60 seconds to 29201 wrap up, save its state for next time, and exit. 29202 With the \f[C]-vP\f[R] flags you will see constant status updates and a 29203 final confirmation of whether or not the graceful shutdown was 29204 successful. 29205 .PP 29206 At any point during the \[dq]Graceful Shutdown\[dq] sequence, a second 29207 \f[C]SIGINT\f[R] or \f[C]Ctrl+C\f[R] will trigger an immediate, 29208 un-graceful exit, which will leave things in a messier state. 29209 Usually a robust recovery will still be possible if using 29210 \f[C]--recover\f[R] mode, otherwise you will need to do a 29211 \f[C]--resync\f[R]. 29212 .PP 29213 If you plan to use Graceful Shutdown mode, it is recommended to use 29214 \f[C]--resilient\f[R] and \f[C]--recover\f[R], and it is important to 29215 NOT use \f[C]--inplace\f[R] (https://rclone.org/docs/#inplace), 29216 otherwise you risk leaving partially-written files on one side, which 29217 may be confused for real files on the next run. 29218 Note also that in the event of an abrupt interruption, a lock file will 29219 be left behind to block concurrent runs. 29220 You will need to delete it before you can proceed with the next run (or 29221 wait for it to expire on its own, if using \f[C]--max-lock\f[R].) 29222 .SS Limitations 29223 .SS Supported backends 29224 .PP 29225 Bisync is considered \f[I]BETA\f[R] and has been tested with the 29226 following backends: - Local filesystem - Google Drive - Dropbox - 29227 OneDrive - S3 - SFTP - Yandex Disk - Crypt 29228 .PP 29229 It has not been fully tested with other services yet. 29230 If it works, or sorta works, please let us know and we\[aq]ll update the 29231 list. 29232 Run the test suite to check for proper operation as described below. 29233 .PP 29234 The first release of \f[C]rclone bisync\f[R] required both underlying 29235 backends to support modification times, and refused to run otherwise. 29236 This limitation has been lifted as of \f[C]v1.66\f[R], as bisync now 29237 supports comparing checksum and/or size instead of (or in addition to) 29238 modtime. 29239 See \f[C]--compare\f[R] for details. 29240 .SS Concurrent modifications 29241 .PP 29242 When using \f[B]Local, FTP or SFTP\f[R] remotes with 29243 \f[C]--inplace\f[R] (https://rclone.org/docs/#inplace), rclone does not 29244 create \f[I]temporary\f[R] files at the destination when copying, and 29245 thus if the connection is lost the created file may be corrupt, which 29246 will likely propagate back to the original path on the next sync, 29247 resulting in data loss. 29248 It is therefore recommended to \f[I]omit\f[R] \f[C]--inplace\f[R]. 29249 .PP 29250 Files that \f[B]change during\f[R] a bisync run may result in data loss. 29251 Prior to \f[C]rclone v1.66\f[R], this was commonly seen in highly 29252 dynamic environments, where the filesystem was getting hammered by 29253 running processes during the sync. 29254 As of \f[C]rclone v1.66\f[R], bisync was redesigned to use a 29255 \[dq]snapshot\[dq] model, greatly reducing the risks from changes during 29256 a sync. 29257 Changes that are not detected during the current sync will now be 29258 detected during the following sync, and will no longer cause the entire 29259 run to throw a critical error. 29260 There is additionally a mechanism to mark files as needing to be 29261 internally rechecked next time, for added safety. 29262 It should therefore no longer be necessary to sync only at quiet times 29263 -- however, note that an error can still occur if a file happens to 29264 change at the exact moment it\[aq]s being read/written by bisync (same 29265 as would happen in \f[C]rclone sync\f[R].) (See also: 29266 \f[C]--ignore-checksum\f[R] (https://rclone.org/docs/#ignore-checksum), 29267 \f[C]--local-no-check-updated\f[R] (https://rclone.org/local/#local-no-check-updated)) 29268 .SS Empty directories 29269 .PP 29270 By default, new/deleted empty directories on one path are \f[I]not\f[R] 29271 propagated to the other side. 29272 This is because bisync (and rclone) natively works on files, not 29273 directories. 29274 However, this can be changed with the \f[C]--create-empty-src-dirs\f[R] 29275 flag, which works in much the same way as in 29276 \f[C]sync\f[R] (https://rclone.org/commands/rclone_sync/) and 29277 \f[C]copy\f[R] (https://rclone.org/commands/rclone_copy/). 29278 When used, empty directories created or deleted on one side will also be 29279 created or deleted on the other side. 29280 The following should be noted: * \f[C]--create-empty-src-dirs\f[R] is 29281 not compatible with \f[C]--remove-empty-dirs\f[R]. 29282 Use only one or the other (or neither). 29283 * It is not recommended to switch back and forth between 29284 \f[C]--create-empty-src-dirs\f[R] and the default (no 29285 \f[C]--create-empty-src-dirs\f[R]) without running \f[C]--resync\f[R]. 29286 This is because it may appear as though all directories (not just the 29287 empty ones) were created/deleted, when actually you\[aq]ve just toggled 29288 between making them visible/invisible to bisync. 29289 It looks scarier than it is, but it\[aq]s still probably best to stick 29290 to one or the other, and use \f[C]--resync\f[R] when you need to switch. 29291 .SS Renamed directories 29292 .PP 29293 By default, renaming a folder on the Path1 side results in deleting all 29294 files on the Path2 side and then copying all files again from Path1 to 29295 Path2. 29296 Bisync sees this as all files in the old directory name as deleted and 29297 all files in the new directory name as new. 29298 .PP 29299 A recommended solution is to use 29300 \f[C]--track-renames\f[R] (https://rclone.org/docs/#track-renames), 29301 which is now supported in bisync as of \f[C]rclone v1.66\f[R]. 29302 Note that \f[C]--track-renames\f[R] is not available during 29303 \f[C]--resync\f[R], as \f[C]--resync\f[R] does not delete anything 29304 (\f[C]--track-renames\f[R] only supports \f[C]sync\f[R], not 29305 \f[C]copy\f[R].) 29306 .PP 29307 Otherwise, the most effective and efficient method of renaming a 29308 directory is to rename it to the same name on both sides. 29309 (As of \f[C]rclone v1.64\f[R], a \f[C]--resync\f[R] is no longer 29310 required after doing so, as bisync will automatically detect that Path1 29311 and Path2 are in agreement.) 29312 .SS \f[C]--fast-list\f[R] used by default 29313 .PP 29314 Unlike most other rclone commands, bisync uses 29315 \f[C]--fast-list\f[R] (https://rclone.org/docs/#fast-list) by default, 29316 for backends that support it. 29317 In many cases this is desirable, however, there are some scenarios in 29318 which bisync could be faster \f[I]without\f[R] \f[C]--fast-list\f[R], 29319 and there is also a known issue concerning Google Drive users with many 29320 empty 29321 directories (https://github.com/artpar/artpar/commit/cbf3d4356135814921382dd3285d859d15d0aa77). 29322 For now, the recommended way to avoid using \f[C]--fast-list\f[R] is to 29323 add \f[C]--disable ListR\f[R] to all bisync commands. 29324 The default behavior may change in a future version. 29325 .SS Case (and unicode) sensitivity 29326 .PP 29327 As of \f[C]v1.66\f[R], case and unicode form differences no longer cause 29328 critical errors, and normalization (when comparing between filesystems) 29329 is handled according to the same flags and defaults as 29330 \f[C]rclone sync\f[R]. 29331 See the following options (all of which are supported by bisync) to 29332 control this behavior more granularly: - 29333 \f[C]--fix-case\f[R] (https://rclone.org/docs/#fix-case) - 29334 \f[C]--ignore-case-sync\f[R] (https://rclone.org/docs/#ignore-case-sync) 29335 - 29336 \f[C]--no-unicode-normalization\f[R] (https://rclone.org/docs/#no-unicode-normalization) 29337 - 29338 \f[C]--local-unicode-normalization\f[R] (https://rclone.org/local/#local-unicode-normalization) 29339 and 29340 \f[C]--local-case-sensitive\f[R] (https://rclone.org/local/#local-case-sensitive) 29341 (caution: these are normally not what you want.) 29342 .PP 29343 Note that in the (probably rare) event that \f[C]--fix-case\f[R] is used 29344 AND a file is new/changed on both sides AND the checksums match AND the 29345 filename case does not match, the Path1 filename is considered the 29346 winner, for the purposes of \f[C]--fix-case\f[R] (Path2 will be renamed 29347 to match it). 29348 .SS Windows support 29349 .PP 29350 Bisync has been tested on Windows 8.1, Windows 10 Pro 64-bit and on 29351 Windows GitHub runners. 29352 .PP 29353 Drive letters are allowed, including drive letters mapped to network 29354 drives (\f[C]rclone bisync J:\[rs]localsync GDrive:\f[R]). 29355 If a drive letter is omitted, the shell current drive is the default. 29356 Drive letters are a single character follows by \f[C]:\f[R], so cloud 29357 names must be more than one character long. 29358 .PP 29359 Absolute paths (with or without a drive letter), and relative paths 29360 (with or without a drive letter) are supported. 29361 .PP 29362 Working directory is created at 29363 \f[C]C:\[rs]Users\[rs]MyLogin\[rs]AppData\[rs]Local\[rs]rclone\[rs]bisync\f[R]. 29364 .PP 29365 Note that bisync output may show a mix of forward \f[C]/\f[R] and back 29366 \f[C]\[rs]\f[R] slashes. 29367 .PP 29368 Be careful of case independent directory and file naming on Windows vs. 29369 case dependent Linux 29370 .SS Filtering 29371 .PP 29372 See filtering documentation (https://rclone.org/filtering/) for how 29373 filter rules are written and interpreted. 29374 .PP 29375 Bisync\[aq]s \f[C]--filters-file\f[R] flag slightly extends the 29376 rclone\[aq]s 29377 --filter-from (https://rclone.org/filtering/#filter-from-read-filtering-patterns-from-a-file) 29378 filtering mechanism. 29379 For a given bisync run you may provide \f[I]only one\f[R] 29380 \f[C]--filters-file\f[R]. 29381 The \f[C]--include*\f[R], \f[C]--exclude*\f[R], and \f[C]--filter\f[R] 29382 flags are also supported. 29383 .SS How to filter directories 29384 .PP 29385 Filtering portions of the directory tree is a critical feature for 29386 synching. 29387 .PP 29388 Examples of directory trees (always beneath the Path1/Path2 root level) 29389 you may want to exclude from your sync: - Directory trees containing 29390 only software build intermediate files. 29391 - Directory trees containing application temporary files and data such 29392 as the Windows \f[C]C:\[rs]Users\[rs]MyLogin\[rs]AppData\[rs]\f[R] tree. 29393 - Directory trees containing files that are large, less important, or 29394 are getting thrashed continuously by ongoing processes. 29395 .PP 29396 On the other hand, there may be only select directories that you 29397 actually want to sync, and exclude all others. 29398 See the Example include-style filters for Windows user directories 29399 below. 29400 .SS Filters file writing guidelines 29401 .IP "1." 3 29402 Begin with excluding directory trees: 29403 .RS 4 29404 .IP \[bu] 2 29405 e.g. 29406 \[ga]- /AppData/\[ga] 29407 .IP \[bu] 2 29408 \f[C]**\f[R] on the end is not necessary. 29409 Once a given directory level is excluded then everything beneath it 29410 won\[aq]t be looked at by rclone. 29411 .IP \[bu] 2 29412 Exclude such directories that are unneeded, are big, dynamically 29413 thrashed, or where there may be access permission issues. 29414 .IP \[bu] 2 29415 Excluding such dirs first will make rclone operations (much) faster. 29416 .IP \[bu] 2 29417 Specific files may also be excluded, as with the Dropbox exclusions 29418 example below. 29419 .RE 29420 .IP "2." 3 29421 Decide if it\[aq]s easier (or cleaner) to: 29422 .RS 4 29423 .IP \[bu] 2 29424 Include select directories and therefore \f[I]exclude everything 29425 else\f[R] -- or -- 29426 .IP \[bu] 2 29427 Exclude select directories and therefore \f[I]include everything 29428 else\f[R] 29429 .RE 29430 .IP "3." 3 29431 Include select directories: 29432 .RS 4 29433 .IP \[bu] 2 29434 Add lines like: \[ga]+ /Documents/PersonalFiles/**\[ga] to select which 29435 directories to include in the sync. 29436 .IP \[bu] 2 29437 \f[C]**\f[R] on the end specifies to include the full depth of the 29438 specified tree. 29439 .IP \[bu] 2 29440 With Include-style filters, files at the Path1/Path2 root are not 29441 included. 29442 They may be included with \[ga]+ /*\[ga]. 29443 .IP \[bu] 2 29444 Place RCLONE_TEST files within these included directory trees. 29445 They will only be looked for in these directory trees. 29446 .IP \[bu] 2 29447 Finish by excluding everything else by adding \[ga]- **\[ga] at the end 29448 of the filters file. 29449 .IP \[bu] 2 29450 Disregard step 4. 29451 .RE 29452 .IP "4." 3 29453 Exclude select directories: 29454 .RS 4 29455 .IP \[bu] 2 29456 Add more lines like in step 1. 29457 For example: \f[C]-/Desktop/tempfiles/\f[R], or \[ga]- 29458 /testdir/\f[C]. Again, a\f[R]**\[ga] on the end is not necessary. 29459 .IP \[bu] 2 29460 Do \f[I]not\f[R] add a \[ga]- **\[ga] in the file. 29461 Without this line, everything will be included that has not been 29462 explicitly excluded. 29463 .IP \[bu] 2 29464 Disregard step 3. 29465 .RE 29466 .PP 29467 A few rules for the syntax of a filter file expanding on filtering 29468 documentation (https://rclone.org/filtering/): 29469 .IP \[bu] 2 29470 Lines may start with spaces and tabs - rclone strips leading whitespace. 29471 .IP \[bu] 2 29472 If the first non-whitespace character is a \f[C]#\f[R] then the line is 29473 a comment and will be ignored. 29474 .IP \[bu] 2 29475 Blank lines are ignored. 29476 .IP \[bu] 2 29477 The first non-whitespace character on a filter line must be a 29478 \f[C]+\f[R] or \f[C]-\f[R]. 29479 .IP \[bu] 2 29480 Exactly 1 space is allowed between the \f[C]+/-\f[R] and the path term. 29481 .IP \[bu] 2 29482 Only forward slashes (\f[C]/\f[R]) are used in path terms, even on 29483 Windows. 29484 .IP \[bu] 2 29485 The rest of the line is taken as the path term. 29486 Trailing whitespace is taken literally, and probably is an error. 29487 .SS Example include-style filters for Windows user directories 29488 .PP 29489 This Windows \f[I]include-style\f[R] example is based on the sync root 29490 (Path1) set to \f[C]C:\[rs]Users\[rs]MyLogin\f[R]. 29491 The strategy is to select specific directories to be synched with a 29492 network drive (Path2). 29493 .IP \[bu] 2 29494 \[ga]- /AppData/\[ga] excludes an entire tree of Windows stored stuff 29495 that need not be synched. 29496 In my case, AppData has >11 GB of stuff I don\[aq]t care about, and 29497 there are some subdirectories beneath AppData that are not accessible to 29498 my user login, resulting in bisync critical aborts. 29499 .IP \[bu] 2 29500 Windows creates cache files starting with both upper and lowercase 29501 \f[C]NTUSER\f[R] at \f[C]C:\[rs]Users\[rs]MyLogin\f[R]. 29502 These files may be dynamic, locked, and are generally \f[I]don\[aq]t 29503 care\f[R]. 29504 .IP \[bu] 2 29505 There are just a few directories with \f[I]my\f[R] data that I do want 29506 synched, in the form of \[ga]+ 29507 /\f[C]. By selecting only the directory trees I want to avoid the dozen plus directories that various apps make at\f[R]C:\[ga]. 29508 .IP \[bu] 2 29509 Include files in the root of the sync point, 29510 \f[C]C:\[rs]Users\[rs]MyLogin\f[R], by adding the \[ga]+ /*\[ga] line. 29511 .IP \[bu] 2 29512 This is an Include-style filters file, therefore it ends with \[ga]- 29513 **\[ga] which excludes everything not explicitly included. 29514 .IP 29515 .nf 29516 \f[C] 29517 - /AppData/ 29518 - NTUSER* 29519 - ntuser* 29520 + /Documents/Family/** 29521 + /Documents/Sketchup/** 29522 + /Documents/Microcapture_Photo/** 29523 + /Documents/Microcapture_Video/** 29524 + /Desktop/** 29525 + /Pictures/** 29526 + /* 29527 - ** 29528 \f[R] 29529 .fi 29530 .PP 29531 Note also that Windows implements several \[dq]library\[dq] links such 29532 as \f[C]C:\[rs]Users\[rs]MyLogin\[rs]My Documents\[rs]My Music\f[R] 29533 pointing to \f[C]C:\[rs]Users\[rs]MyLogin\[rs]Music\f[R]. 29534 rclone sees these as links, so you must add \f[C]--links\f[R] to the 29535 bisync command line if you which to follow these links. 29536 I find that I get permission errors in trying to follow the links, so I 29537 don\[aq]t include the rclone \f[C]--links\f[R] flag, but then you get 29538 lots of \f[C]Can\[aq]t follow symlink\&...\f[R] noise from rclone about 29539 not following the links. 29540 This noise can be quashed by adding \f[C]--quiet\f[R] to the bisync 29541 command line. 29542 .SS Example exclude-style filters files for use with Dropbox 29543 .IP \[bu] 2 29544 Dropbox disallows synching the listed temporary and configuration/data 29545 files. 29546 The \[ga]- \[ga] filters exclude these files where ever they may occur 29547 in the sync tree. 29548 Consider adding similar exclusions for file types you don\[aq]t need to 29549 sync, such as core dump and software build files. 29550 .IP \[bu] 2 29551 bisync testing creates \f[C]/testdir/\f[R] at the top level of the sync 29552 tree, and usually deletes the tree after the test. 29553 If a normal sync should run while the \f[C]/testdir/\f[R] tree exists 29554 the \f[C]--check-access\f[R] phase may fail due to unbalanced 29555 RCLONE_TEST files. 29556 The \[ga]- /testdir/\[ga] filter blocks this tree from being synched. 29557 You don\[aq]t need this exclusion if you are not doing bisync 29558 development testing. 29559 .IP \[bu] 2 29560 Everything else beneath the Path1/Path2 root will be synched. 29561 .IP \[bu] 2 29562 RCLONE_TEST files may be placed anywhere within the tree, including the 29563 root. 29564 .SS Example filters file for Dropbox 29565 .IP 29566 .nf 29567 \f[C] 29568 # Filter file for use with bisync 29569 # See https://rclone.org/filtering/ for filtering rules 29570 # NOTICE: If you make changes to this file you MUST do a --resync run. 29571 # Run with --dry-run to see what changes will be made. 29572 29573 # Dropbox won\[aq]t sync some files so filter them away here. 29574 # See https://help.dropbox.com/installs-integrations/sync-uploads/files-not-syncing 29575 - .dropbox.attr 29576 - \[ti]*.tmp 29577 - \[ti]$* 29578 - .\[ti]* 29579 - desktop.ini 29580 - .dropbox 29581 29582 # Used for bisync testing, so excluded from normal runs 29583 - /testdir/ 29584 29585 # Other example filters 29586 #- /TiBU/ 29587 #- /Photos/ 29588 \f[R] 29589 .fi 29590 .SS How --check-access handles filters 29591 .PP 29592 At the start of a bisync run, listings are gathered for Path1 and Path2 29593 while using the user\[aq]s \f[C]--filters-file\f[R]. 29594 During the check access phase, bisync scans these listings for 29595 \f[C]RCLONE_TEST\f[R] files. 29596 Any \f[C]RCLONE_TEST\f[R] files hidden by the \f[C]--filters-file\f[R] 29597 are \f[I]not\f[R] in the listings and thus not checked during the check 29598 access phase. 29599 .SS Troubleshooting 29600 .SS Reading bisync logs 29601 .PP 29602 Here are two normal runs. 29603 The first one has a newer file on the remote. 29604 The second has no deltas between local and remote. 29605 .IP 29606 .nf 29607 \f[C] 29608 2021/05/16 00:24:38 INFO : Synching Path1 \[dq]/path/to/local/tree/\[dq] with Path2 \[dq]dropbox:/\[dq] 29609 2021/05/16 00:24:38 INFO : Path1 checking for diffs 29610 2021/05/16 00:24:38 INFO : - Path1 File is new - file.txt 29611 2021/05/16 00:24:38 INFO : Path1: 1 changes: 1 new, 0 newer, 0 older, 0 deleted 29612 2021/05/16 00:24:38 INFO : Path2 checking for diffs 29613 2021/05/16 00:24:38 INFO : Applying changes 29614 2021/05/16 00:24:38 INFO : - Path1 Queue copy to Path2 - dropbox:/file.txt 29615 2021/05/16 00:24:38 INFO : - Path1 Do queued copies to - Path2 29616 2021/05/16 00:24:38 INFO : Updating listings 29617 2021/05/16 00:24:38 INFO : Validating listings for Path1 \[dq]/path/to/local/tree/\[dq] vs Path2 \[dq]dropbox:/\[dq] 29618 2021/05/16 00:24:38 INFO : Bisync successful 29619 29620 2021/05/16 00:36:52 INFO : Synching Path1 \[dq]/path/to/local/tree/\[dq] with Path2 \[dq]dropbox:/\[dq] 29621 2021/05/16 00:36:52 INFO : Path1 checking for diffs 29622 2021/05/16 00:36:52 INFO : Path2 checking for diffs 29623 2021/05/16 00:36:52 INFO : No changes found 29624 2021/05/16 00:36:52 INFO : Updating listings 29625 2021/05/16 00:36:52 INFO : Validating listings for Path1 \[dq]/path/to/local/tree/\[dq] vs Path2 \[dq]dropbox:/\[dq] 29626 2021/05/16 00:36:52 INFO : Bisync successful 29627 \f[R] 29628 .fi 29629 .SS Dry run oddity 29630 .PP 29631 The \f[C]--dry-run\f[R] messages may indicate that it would try to 29632 delete some files. 29633 For example, if a file is new on Path2 and does not exist on Path1 then 29634 it would normally be copied to Path1, but with \f[C]--dry-run\f[R] 29635 enabled those copies don\[aq]t happen, which leads to the attempted 29636 delete on Path2, blocked again by --dry-run: 29637 \f[C]... Not deleting as --dry-run\f[R]. 29638 .PP 29639 This whole confusing situation is an artifact of the \f[C]--dry-run\f[R] 29640 flag. 29641 Scrutinize the proposed deletes carefully, and if the files would have 29642 been copied to Path1 then the threatened deletes on Path2 may be 29643 disregarded. 29644 .SS Retries 29645 .PP 29646 Rclone has built-in retries. 29647 If you run with \f[C]--verbose\f[R] you\[aq]ll see error and retry 29648 messages such as shown below. 29649 This is usually not a bug. 29650 If at the end of the run, you see \f[C]Bisync successful\f[R] and not 29651 \f[C]Bisync critical error\f[R] or \f[C]Bisync aborted\f[R] then the run 29652 was successful, and you can ignore the error messages. 29653 .PP 29654 The following run shows an intermittent fail. 29655 Lines \f[I]5\f[R] and _6- are low-level messages. 29656 Line \f[I]6\f[R] is a bubbled-up \f[I]warning\f[R] message, conveying 29657 the error. 29658 Rclone normally retries failing commands, so there may be numerous such 29659 messages in the log. 29660 .PP 29661 Since there are no final error/warning messages on line \f[I]7\f[R], 29662 rclone has recovered from failure after a retry, and the overall sync 29663 was successful. 29664 .IP 29665 .nf 29666 \f[C] 29667 1: 2021/05/14 00:44:12 INFO : Synching Path1 \[dq]/path/to/local/tree\[dq] with Path2 \[dq]dropbox:\[dq] 29668 2: 2021/05/14 00:44:12 INFO : Path1 checking for diffs 29669 3: 2021/05/14 00:44:12 INFO : Path2 checking for diffs 29670 4: 2021/05/14 00:44:12 INFO : Path2: 113 changes: 22 new, 0 newer, 0 older, 91 deleted 29671 5: 2021/05/14 00:44:12 ERROR : /path/to/local/tree/objects/af: error listing: unexpected end of JSON input 29672 6: 2021/05/14 00:44:12 NOTICE: WARNING listing try 1 failed. - dropbox: 29673 7: 2021/05/14 00:44:12 INFO : Bisync successful 29674 \f[R] 29675 .fi 29676 .PP 29677 This log shows a \f[I]Critical failure\f[R] which requires a 29678 \f[C]--resync\f[R] to recover from. 29679 See the Runtime Error Handling section. 29680 .IP 29681 .nf 29682 \f[C] 29683 2021/05/12 00:49:40 INFO : Google drive root \[aq]\[aq]: Waiting for checks to finish 29684 2021/05/12 00:49:40 INFO : Google drive root \[aq]\[aq]: Waiting for transfers to finish 29685 2021/05/12 00:49:40 INFO : Google drive root \[aq]\[aq]: not deleting files as there were IO errors 29686 2021/05/12 00:49:40 ERROR : Attempt 3/3 failed with 3 errors and: not deleting files as there were IO errors 29687 2021/05/12 00:49:40 ERROR : Failed to sync: not deleting files as there were IO errors 29688 2021/05/12 00:49:40 NOTICE: WARNING rclone sync try 3 failed. - /path/to/local/tree/ 29689 2021/05/12 00:49:40 ERROR : Bisync aborted. Must run --resync to recover. 29690 \f[R] 29691 .fi 29692 .SS Denied downloads of \[dq]infected\[dq] or \[dq]abusive\[dq] files 29693 .PP 29694 Google Drive has a filter for certain file types (\f[C].exe\f[R], 29695 \f[C].apk\f[R], et cetera) that by default cannot be copied from Google 29696 Drive to the local filesystem. 29697 If you are having problems, run with \f[C]--verbose\f[R] to see 29698 specifically which files are generating complaints. 29699 If the error is 29700 \f[C]This file has been identified as malware or spam and cannot be downloaded\f[R], 29701 consider using the flag 29702 --drive-acknowledge-abuse (https://rclone.org/drive/#drive-acknowledge-abuse). 29703 .SS Google Docs (and other files of unknown size) 29704 .PP 29705 As of \f[C]v1.66\f[R], Google 29706 Docs (https://rclone.org/drive/#import-export-of-google-documents) 29707 (including Google Sheets, Slides, etc.) are now supported in bisync, 29708 subject to the same options, defaults, and limitations as in 29709 \f[C]rclone sync\f[R]. 29710 When bisyncing drive with non-drive backends, the drive -> non-drive 29711 direction is controlled by 29712 \f[C]--drive-export-formats\f[R] (https://rclone.org/drive/#drive-export-formats) 29713 (default \f[C]\[dq]docx,xlsx,pptx,svg\[dq]\f[R]) and the non-drive -> 29714 drive direction is controlled by 29715 \f[C]--drive-import-formats\f[R] (https://rclone.org/drive/#drive-import-formats) 29716 (default none.) 29717 .PP 29718 For example, with the default export/import formats, a Google Sheet on 29719 the drive side will be synced to an \f[C].xlsx\f[R] file on the 29720 non-drive side. 29721 In the reverse direction, \f[C].xlsx\f[R] files with filenames that 29722 match an existing Google Sheet will be synced to that Google Sheet, 29723 while \f[C].xlsx\f[R] files that do NOT match an existing Google Sheet 29724 will be copied to drive as normal \f[C].xlsx\f[R] files (without 29725 conversion to Sheets, although the Google Drive web browser UI may still 29726 give you the option to open it as one.) 29727 .PP 29728 If \f[C]--drive-import-formats\f[R] is set (it\[aq]s not, by default), 29729 then all of the specified formats will be converted to Google Docs, if 29730 there is no existing Google Doc with a matching name. 29731 Caution: such conversion can be quite lossy, and in most cases it\[aq]s 29732 probably not what you want! 29733 .PP 29734 To bisync Google Docs as URL shortcut links (in a manner similar to 29735 \[dq]Drive for Desktop\[dq]), use: \f[C]--drive-export-formats url\f[R] 29736 (or 29737 alternatives (https://rclone.org/drive/#exportformats:~:text=available%20Google%20Documents.-,Extension,macOS,-Standard%20options).) 29738 .PP 29739 Note that these link files cannot be edited on the non-drive side -- you 29740 will get errors if you try to sync an edited link file back to drive. 29741 They CAN be deleted (it will result in deleting the corresponding Google 29742 Doc.) If you create a \f[C].url\f[R] file on the non-drive side that 29743 does not match an existing Google Doc, bisyncing it will just result in 29744 copying the literal \f[C].url\f[R] file over to drive (no Google Doc 29745 will be created.) So, as a general rule of thumb, think of them as 29746 read-only placeholders on the non-drive side, and make all your changes 29747 on the drive side. 29748 .PP 29749 Likewise, even with other export-formats, it is best to only move/rename 29750 Google Docs on the drive side. 29751 This is because otherwise, bisync will interpret this as a file deleted 29752 and another created, and accordingly, it will delete the Google Doc and 29753 create a new file at the new path. 29754 (Whether or not that new file is a Google Doc depends on 29755 \f[C]--drive-import-formats\f[R].) 29756 .PP 29757 Lastly, take note that all Google Docs on the drive side have a size of 29758 \f[C]-1\f[R] and no checksum. 29759 Therefore, they cannot be reliably synced with the \f[C]--checksum\f[R] 29760 or \f[C]--size-only\f[R] flags. 29761 (To be exact: they will still get created/deleted, and bisync\[aq]s 29762 delta engine will notice changes and queue them for syncing, but the 29763 underlying sync function will consider them identical and skip them.) To 29764 work around this, use the default (modtime and size) instead of 29765 \f[C]--checksum\f[R] or \f[C]--size-only\f[R]. 29766 .PP 29767 To ignore Google Docs entirely, use 29768 \f[C]--drive-skip-gdocs\f[R] (https://rclone.org/drive/#drive-skip-gdocs). 29769 .SS Usage examples 29770 .SS Cron 29771 .PP 29772 Rclone does not yet have a built-in capability to monitor the local file 29773 system for changes and must be blindly run periodically. 29774 On Windows this can be done using a \f[I]Task Scheduler\f[R], on Linux 29775 you can use \f[I]Cron\f[R] which is described below. 29776 .PP 29777 The 1st example runs a sync every 5 minutes between a local directory 29778 and an OwnCloud server, with output logged to a runlog file: 29779 .IP 29780 .nf 29781 \f[C] 29782 # Minute (0-59) 29783 # Hour (0-23) 29784 # Day of Month (1-31) 29785 # Month (1-12 or Jan-Dec) 29786 # Day of Week (0-6 or Sun-Sat) 29787 # Command 29788 */5 * * * * /path/to/rclone bisync /local/files MyCloud: --check-access --filters-file /path/to/bysync-filters.txt --log-file /path/to//bisync.log 29789 \f[R] 29790 .fi 29791 .PP 29792 See crontab 29793 syntax (https://www.man7.org/linux/man-pages/man1/crontab.1p.html#INPUT_FILES) 29794 for the details of crontab time interval expressions. 29795 .PP 29796 If you run \f[C]rclone bisync\f[R] as a cron job, redirect stdout/stderr 29797 to a file. 29798 The 2nd example runs a sync to Dropbox every hour and logs all stdout 29799 (via the \f[C]>>\f[R]) and stderr (via \f[C]2>&1\f[R]) to a log file. 29800 .IP 29801 .nf 29802 \f[C] 29803 0 * * * * /path/to/rclone bisync /path/to/local/dropbox Dropbox: --check-access --filters-file /home/user/filters.txt >> /path/to/logs/dropbox-run.log 2>&1 29804 \f[R] 29805 .fi 29806 .SS Sharing an encrypted folder tree between hosts 29807 .PP 29808 bisync can keep a local folder in sync with a cloud service, but what if 29809 you have some highly sensitive files to be synched? 29810 .PP 29811 Usage of a cloud service is for exchanging both routine and sensitive 29812 personal files between one\[aq]s home network, one\[aq]s personal 29813 notebook when on the road, and with one\[aq]s work computer. 29814 The routine data is not sensitive. 29815 For the sensitive data, configure an rclone crypt 29816 remote (https://rclone.org/crypt/) to point to a subdirectory within the 29817 local disk tree that is bisync\[aq]d to Dropbox, and then set up an 29818 bisync for this local crypt directory to a directory outside of the main 29819 sync tree. 29820 .SS Linux server setup 29821 .IP \[bu] 2 29822 \f[C]/path/to/DBoxroot\f[R] is the root of my local sync tree. 29823 There are numerous subdirectories. 29824 .IP \[bu] 2 29825 \f[C]/path/to/DBoxroot/crypt\f[R] is the root subdirectory for files 29826 that are encrypted. 29827 This local directory target is setup as an rclone crypt remote named 29828 \f[C]Dropcrypt:\f[R]. 29829 See rclone.conf snippet below. 29830 .IP \[bu] 2 29831 \f[C]/path/to/my/unencrypted/files\f[R] is the root of my sensitive 29832 files - not encrypted, not within the tree synched to Dropbox. 29833 .IP \[bu] 2 29834 To sync my local unencrypted files with the encrypted Dropbox versions I 29835 manually run \f[C]bisync /path/to/my/unencrypted/files DropCrypt:\f[R]. 29836 This step could be bundled into a script to run before and after the 29837 full Dropbox tree sync in the last step, thus actively keeping the 29838 sensitive files in sync. 29839 .IP \[bu] 2 29840 \f[C]bisync /path/to/DBoxroot Dropbox:\f[R] runs periodically via cron, 29841 keeping my full local sync tree in sync with Dropbox. 29842 .SS Windows notebook setup 29843 .IP \[bu] 2 29844 The Dropbox client runs keeping the local tree 29845 \f[C]C:\[rs]Users\[rs]MyLogin\[rs]Dropbox\f[R] always in sync with 29846 Dropbox. 29847 I could have used \f[C]rclone bisync\f[R] instead. 29848 .IP \[bu] 2 29849 A separate directory tree at 29850 \f[C]C:\[rs]Users\[rs]MyLogin\[rs]Documents\[rs]DropLocal\f[R] hosts the 29851 tree of unencrypted files/folders. 29852 .IP \[bu] 2 29853 To sync my local unencrypted files with the encrypted Dropbox versions I 29854 manually run the following command: 29855 \f[C]rclone bisync C:\[rs]Users\[rs]MyLogin\[rs]Documents\[rs]DropLocal Dropcrypt:\f[R]. 29856 .IP \[bu] 2 29857 The Dropbox client then syncs the changes with Dropbox. 29858 .SS rclone.conf snippet 29859 .IP 29860 .nf 29861 \f[C] 29862 [Dropbox] 29863 type = dropbox 29864 \&... 29865 29866 [Dropcrypt] 29867 type = crypt 29868 remote = /path/to/DBoxroot/crypt # on the Linux server 29869 remote = C:\[rs]Users\[rs]MyLogin\[rs]Dropbox\[rs]crypt # on the Windows notebook 29870 filename_encryption = standard 29871 directory_name_encryption = true 29872 password = ... 29873 \&... 29874 \f[R] 29875 .fi 29876 .SS Testing 29877 .PP 29878 You should read this section only if you are developing for rclone. 29879 You need to have rclone source code locally to work with bisync tests. 29880 .PP 29881 Bisync has a dedicated test framework implemented in the 29882 \f[C]bisync_test.go\f[R] file located in the rclone source tree. 29883 The test suite is based on the \f[C]go test\f[R] command. 29884 Series of tests are stored in subdirectories below the 29885 \f[C]cmd/bisync/testdata\f[R] directory. 29886 Individual tests can be invoked by their directory name, e.g. 29887 \f[C]go test . -case basic -remote local -remote2 gdrive: -v\f[R] 29888 .PP 29889 Tests will make a temporary folder on remote and purge it afterwards. 29890 If during test run there are intermittent errors and rclone retries, 29891 these errors will be captured and flagged as invalid MISCOMPAREs. 29892 Rerunning the test will let it pass. 29893 Consider such failures as noise. 29894 .SS Test command syntax 29895 .IP 29896 .nf 29897 \f[C] 29898 usage: go test ./cmd/bisync [options...] 29899 29900 Options: 29901 -case NAME Name(s) of the test case(s) to run. Multiple names should 29902 be separated by commas. You can remove the \[ga]test_\[ga] prefix 29903 and replace \[ga]_\[ga] by \[ga]-\[ga] in test name for convenience. 29904 If not \[ga]all\[ga], the name(s) should map to a directory under 29905 \[ga]./cmd/bisync/testdata\[ga]. 29906 Use \[ga]all\[ga] to run all tests (default: all) 29907 -remote PATH1 \[ga]local\[ga] or name of cloud service with \[ga]:\[ga] (default: local) 29908 -remote2 PATH2 \[ga]local\[ga] or name of cloud service with \[ga]:\[ga] (default: local) 29909 -no-compare Disable comparing test results with the golden directory 29910 (default: compare) 29911 -no-cleanup Disable cleanup of Path1 and Path2 testdirs. 29912 Useful for troubleshooting. (default: cleanup) 29913 -golden Store results in the golden directory (default: false) 29914 This flag can be used with multiple tests. 29915 -debug Print debug messages 29916 -stop-at NUM Stop test after given step number. (default: run to the end) 29917 Implies \[ga]-no-compare\[ga] and \[ga]-no-cleanup\[ga], if the test really 29918 ends prematurely. Only meaningful for a single test case. 29919 -refresh-times Force refreshing the target modtime, useful for Dropbox 29920 (default: false) 29921 -verbose Run tests verbosely 29922 \f[R] 29923 .fi 29924 .PP 29925 Note: unlike rclone flags which must be prefixed by double dash 29926 (\f[C]--\f[R]), the test command flags can be equally prefixed by a 29927 single \f[C]-\f[R] or double dash. 29928 .SS Running tests 29929 .IP \[bu] 2 29930 \f[C]go test . -case basic -remote local -remote2 local\f[R] runs the 29931 \f[C]test_basic\f[R] test case using only the local filesystem, synching 29932 one local directory with another local directory. 29933 Test script output is to the console, while commands within scenario.txt 29934 have their output sent to the \f[C].../workdir/test.log\f[R] file, which 29935 is finally compared to the golden copy. 29936 .IP \[bu] 2 29937 The first argument after \f[C]go test\f[R] should be a relative name of 29938 the directory containing bisync source code. 29939 If you run tests right from there, the argument will be \f[C].\f[R] 29940 (current directory) as in most examples below. 29941 If you run bisync tests from the rclone source directory, the command 29942 should be \f[C]go test ./cmd/bisync ...\f[R]. 29943 .IP \[bu] 2 29944 The test engine will mangle rclone output to ensure comparability with 29945 golden listings and logs. 29946 .IP \[bu] 2 29947 Test scenarios are located in \f[C]./cmd/bisync/testdata\f[R]. 29948 The test \f[C]-case\f[R] argument should match the full name of a 29949 subdirectory under that directory. 29950 Every test subdirectory name on disk must start with \f[C]test_\f[R], 29951 this prefix can be omitted on command line for brevity. 29952 Also, underscores in the name can be replaced by dashes for convenience. 29953 .IP \[bu] 2 29954 \f[C]go test . -remote local -remote2 local -case all\f[R] runs all 29955 tests. 29956 .IP \[bu] 2 29957 Path1 and Path2 may either be the keyword \f[C]local\f[R] or may be 29958 names of configured cloud services. 29959 \f[C]go test . -remote gdrive: -remote2 dropbox: -case basic\f[R] will 29960 run the test between these two services, without transferring any files 29961 to the local filesystem. 29962 .IP \[bu] 2 29963 Test run stdout and stderr console output may be directed to a file, 29964 e.g. 29965 \f[C]go test . -remote gdrive: -remote2 local -case all > runlog.txt 2>&1\f[R] 29966 .SS Test execution flow 29967 .IP "1." 3 29968 The base setup in the \f[C]initial\f[R] directory of the testcase is 29969 applied on the Path1 and Path2 filesystems (via rclone copy the initial 29970 directory to Path1, then rclone sync Path1 to Path2). 29971 .IP "2." 3 29972 The commands in the scenario.txt file are applied, with output directed 29973 to the \f[C]test.log\f[R] file in the test working directory. 29974 Typically, the first actual command in the \f[C]scenario.txt\f[R] file 29975 is to do a \f[C]--resync\f[R], which establishes the baseline 29976 \f[C]{...}.path1.lst\f[R] and \f[C]{...}.path2.lst\f[R] files in the 29977 test working directory (\f[C].../workdir/\f[R] relative to the temporary 29978 test directory). 29979 Various commands and listing snapshots are done within the test. 29980 .IP "3." 3 29981 Finally, the contents of the test working directory are compared to the 29982 contents of the testcase\[aq]s golden directory. 29983 .SS Notes about testing 29984 .IP \[bu] 2 29985 Test cases are in individual directories beneath 29986 \f[C]./cmd/bisync/testdata\f[R]. 29987 A command line reference to a test is understood to reference a 29988 directory beneath \f[C]testdata\f[R]. 29989 For example, 29990 \f[C]go test ./cmd/bisync -case dry-run -remote gdrive: -remote2 local\f[R] 29991 refers to the test case in \f[C]./cmd/bisync/testdata/test_dry_run\f[R]. 29992 .IP \[bu] 2 29993 The test working directory is located at \f[C].../workdir\f[R] relative 29994 to a temporary test directory, usually under \f[C]/tmp\f[R] on Linux. 29995 .IP \[bu] 2 29996 The local test sync tree is created at a temporary directory named like 29997 \f[C]bisync.XXX\f[R] under system temporary directory. 29998 .IP \[bu] 2 29999 The remote test sync tree is located at a temporary directory under 30000 \f[C]<remote:>/bisync.XXX/\f[R]. 30001 .IP \[bu] 2 30002 \f[C]path1\f[R] and/or \f[C]path2\f[R] subdirectories are created in a 30003 temporary directory under the respective local or cloud test remote. 30004 .IP \[bu] 2 30005 By default, the Path1 and Path2 test dirs and workdir will be deleted 30006 after each test run. 30007 The \f[C]-no-cleanup\f[R] flag disables purging these directories when 30008 validating and debugging a given test. 30009 These directories will be flushed before running another test, 30010 independent of the \f[C]-no-cleanup\f[R] usage. 30011 .IP \[bu] 2 30012 You will likely want to add \[ga]- 30013 /testdir/\f[C]to your normal bisync\f[R]--filters-file\f[C]so that normal syncs do not attempt to sync the test temporary directories, which may have\f[R]RCLONE_TEST\f[C]miscompares in some testcases which would otherwise trip the\f[R]--check-access\f[C]system. The\f[R]--check-access\f[C]mechanism is hard-coded to ignore\f[R]RCLONE_TEST\f[C]files beneath\f[R]bisync/testdata\[ga], 30014 so the test cases may reside on the synched tree even if there are check 30015 file mismatches in the test tree. 30016 .IP \[bu] 2 30017 Some Dropbox tests can fail, notably printing the following message: 30018 \f[C]src and dst identical but can\[aq]t set mod time without deleting and re-uploading\f[R] 30019 This is expected and happens due to the way Dropbox handles modification 30020 times. 30021 You should use the \f[C]-refresh-times\f[R] test flag to make up for 30022 this. 30023 .IP \[bu] 2 30024 If Dropbox tests hit request limit for you and print error message 30025 \f[C]too_many_requests/...: Too many requests or write operations.\f[R] 30026 then follow the Dropbox App ID 30027 instructions (https://rclone.org/dropbox/#get-your-own-dropbox-app-id). 30028 .SS Updating golden results 30029 .PP 30030 Sometimes even a slight change in the bisync source can cause little 30031 changes spread around many log files. 30032 Updating them manually would be a nightmare. 30033 .PP 30034 The \f[C]-golden\f[R] flag will store the \f[C]test.log\f[R] and 30035 \f[C]*.lst\f[R] listings from each test case into respective golden 30036 directories. 30037 Golden results will automatically contain generic strings instead of 30038 local or cloud paths which means that they should match when run with a 30039 different cloud service. 30040 .PP 30041 Your normal workflow might be as follows: 1. 30042 Git-clone the rclone sources locally 2. 30043 Modify bisync source and check that it builds 3. 30044 Run the whole test suite \f[C]go test ./cmd/bisync -remote local\f[R] 4. 30045 If some tests show log difference, recheck them individually, e.g.: 30046 \f[C]go test ./cmd/bisync -remote local -case basic\f[R] 5. 30047 If you are convinced with the difference, goldenize all tests at once: 30048 \f[C]go test ./cmd/bisync -remote local -golden\f[R] 6. 30049 Use word diff: \f[C]git diff --word-diff ./cmd/bisync/testdata/\f[R]. 30050 Please note that normal line-level diff is generally useless here. 30051 7. 30052 Check the difference \f[I]carefully\f[R]! 8. 30053 Commit the change (\f[C]git commit\f[R]) \f[I]only\f[R] if you are sure. 30054 If unsure, save your code changes then wipe the log diffs from git: 30055 \f[C]git reset [--hard]\f[R]. 30056 .SS Structure of test scenarios 30057 .IP \[bu] 2 30058 \f[C]<testname>/initial/\f[R] contains a tree of files that will be set 30059 as the initial condition on both Path1 and Path2 testdirs. 30060 .IP \[bu] 2 30061 \f[C]<testname>/modfiles/\f[R] contains files that will be used to 30062 modify the Path1 and/or Path2 filesystems. 30063 .IP \[bu] 2 30064 \f[C]<testname>/golden/\f[R] contains the expected content of the test 30065 working directory (\f[C]workdir\f[R]) at the completion of the testcase. 30066 .IP \[bu] 2 30067 \f[C]<testname>/scenario.txt\f[R] contains the body of the test, in the 30068 form of various commands to modify files, run bisync, and snapshot 30069 listings. 30070 Output from these commands is captured to \f[C].../workdir/test.log\f[R] 30071 for comparison to the golden files. 30072 .SS Supported test commands 30073 .IP \[bu] 2 30074 \f[C]test <some message>\f[R] Print the line to the console and to the 30075 \f[C]test.log\f[R]: 30076 \f[C]test sync is working correctly with options x, y, z\f[R] 30077 .IP \[bu] 2 30078 \f[C]copy-listings <prefix>\f[R] Save a copy of all \f[C].lst\f[R] 30079 listings in the test working directory with the specified prefix: 30080 \f[C]save-listings exclude-pass-run\f[R] 30081 .IP \[bu] 2 30082 \f[C]move-listings <prefix>\f[R] Similar to \f[C]copy-listings\f[R] but 30083 removes the source 30084 .IP \[bu] 2 30085 \f[C]purge-children <dir>\f[R] This will delete all child files and 30086 purge all child subdirs under given directory but keep the parent 30087 intact. 30088 This behavior is important for tests with Google Drive because removing 30089 and re-creating the parent would change its ID. 30090 .IP \[bu] 2 30091 \f[C]delete-file <file>\f[R] Delete a single file. 30092 .IP \[bu] 2 30093 \f[C]delete-glob <dir> <pattern>\f[R] Delete a group of files located 30094 one level deep in the given directory with names matching a given glob 30095 pattern. 30096 .IP \[bu] 2 30097 \f[C]touch-glob YYYY-MM-DD <dir> <pattern>\f[R] Change modification time 30098 on a group of files. 30099 .IP \[bu] 2 30100 \f[C]touch-copy YYYY-MM-DD <source-file> <dest-dir>\f[R] Change file 30101 modification time then copy it to destination. 30102 .IP \[bu] 2 30103 \f[C]copy-file <source-file> <dest-dir>\f[R] Copy a single file to given 30104 directory. 30105 .IP \[bu] 2 30106 \f[C]copy-as <source-file> <dest-file>\f[R] Similar to above but 30107 destination must include both directory and the new file name at 30108 destination. 30109 .IP \[bu] 2 30110 \f[C]copy-dir <src> <dst>\f[R] and \f[C]sync-dir <src> <dst>\f[R] 30111 Copy/sync a directory. 30112 Equivalent of \f[C]rclone copy\f[R] and \f[C]rclone sync\f[R]. 30113 .IP \[bu] 2 30114 \f[C]list-dirs <dir>\f[R] Equivalent to 30115 \f[C]rclone lsf -R --dirs-only <dir>\f[R] 30116 .IP \[bu] 2 30117 \f[C]bisync [options]\f[R] Runs bisync against \f[C]-remote\f[R] and 30118 \f[C]-remote2\f[R]. 30119 .SS Supported substitution terms 30120 .IP \[bu] 2 30121 \f[C]{testdir/}\f[R] - the root dir of the testcase 30122 .IP \[bu] 2 30123 \f[C]{datadir/}\f[R] - the \f[C]modfiles\f[R] dir under the testcase 30124 root 30125 .IP \[bu] 2 30126 \f[C]{workdir/}\f[R] - the temporary test working directory 30127 .IP \[bu] 2 30128 \f[C]{path1/}\f[R] - the root of the Path1 test directory tree 30129 .IP \[bu] 2 30130 \f[C]{path2/}\f[R] - the root of the Path2 test directory tree 30131 .IP \[bu] 2 30132 \f[C]{session}\f[R] - base name of the test listings 30133 .IP \[bu] 2 30134 \f[C]{/}\f[R] - OS-specific path separator 30135 .IP \[bu] 2 30136 \f[C]{spc}\f[R], \f[C]{tab}\f[R], \f[C]{eol}\f[R] - whitespace 30137 .IP \[bu] 2 30138 \f[C]{chr:HH}\f[R] - raw byte with given hexadecimal code 30139 .PP 30140 Substitution results of the terms named like \f[C]{dir/}\f[R] will end 30141 with \f[C]/\f[R] (or backslash on Windows), so it is not necessary to 30142 include slash in the usage, for example 30143 \f[C]delete-file {path1/}file1.txt\f[R]. 30144 .SS Benchmarks 30145 .PP 30146 \f[I]This section is work in progress.\f[R] 30147 .PP 30148 Here are a few data points for scale, execution times, and memory usage. 30149 .PP 30150 The first set of data was taken between a local disk to Dropbox. 30151 The speedtest.net (https://speedtest.net) download speed was \[ti]170 30152 Mbps, and upload speed was \[ti]10 Mbps. 30153 500 files (\[ti]9.5 MB each) had been already synched. 30154 50 files were added in a new directory, each \[ti]9.5 MB, \[ti]475 MB 30155 total. 30156 .PP 30157 .TS 30158 tab(@); 30159 lw(23.8n) lw(35.0n) lw(11.2n). 30160 T{ 30161 Change 30162 T}@T{ 30163 Operations and times 30164 T}@T{ 30165 Overall run time 30166 T} 30167 _ 30168 T{ 30169 500 files synched (nothing to move) 30170 T}@T{ 30171 1x listings for Path1 & Path2 30172 T}@T{ 30173 1.5 sec 30174 T} 30175 T{ 30176 500 files synched with --check-access 30177 T}@T{ 30178 1x listings for Path1 & Path2 30179 T}@T{ 30180 1.5 sec 30181 T} 30182 T{ 30183 50 new files on remote 30184 T}@T{ 30185 Queued 50 copies down: 27 sec 30186 T}@T{ 30187 29 sec 30188 T} 30189 T{ 30190 Moved local dir 30191 T}@T{ 30192 Queued 50 copies up: 410 sec, 50 deletes up: 9 sec 30193 T}@T{ 30194 421 sec 30195 T} 30196 T{ 30197 Moved remote dir 30198 T}@T{ 30199 Queued 50 copies down: 31 sec, 50 deletes down: <1 sec 30200 T}@T{ 30201 33 sec 30202 T} 30203 T{ 30204 Delete local dir 30205 T}@T{ 30206 Queued 50 deletes up: 9 sec 30207 T}@T{ 30208 13 sec 30209 T} 30210 .TE 30211 .PP 30212 This next data is from a user\[aq]s application. 30213 They had \[ti]400GB of data over 1.96 million files being sync\[aq]ed 30214 between a Windows local disk and some remote cloud. 30215 The file full path length was on average 35 characters (which factors 30216 into load time and RAM required). 30217 .IP \[bu] 2 30218 Loading the prior listing into memory (1.96 million files, listing file 30219 size 140 MB) took \[ti]30 sec and occupied about 1 GB of RAM. 30220 .IP \[bu] 2 30221 Getting a fresh listing of the local file system (producing the 140 MB 30222 output file) took about XXX sec. 30223 .IP \[bu] 2 30224 Getting a fresh listing of the remote file system (producing the 140 MB 30225 output file) took about XXX sec. 30226 The network download speed was measured at XXX Mb/s. 30227 .IP \[bu] 2 30228 Once the prior and current Path1 and Path2 listings were loaded (a total 30229 of four to be loaded, two at a time), determining the deltas was pretty 30230 quick (a few seconds for this test case), and the transfer time for any 30231 files to be copied was dominated by the network bandwidth. 30232 .SS References 30233 .PP 30234 rclone\[aq]s bisync implementation was derived from the 30235 rclonesync-V2 (https://github.com/cjnaz/rclonesync-V2) project, 30236 including documentation and test mechanisms, with 30237 [\[at]cjnaz](https://github.com/cjnaz)\[aq]s full support and 30238 encouragement. 30239 .PP 30240 \f[C]rclone bisync\f[R] is similar in nature to a range of other 30241 projects: 30242 .IP \[bu] 2 30243 unison (https://github.com/bcpierce00/unison) 30244 .IP \[bu] 2 30245 syncthing (https://github.com/syncthing/syncthing) 30246 .IP \[bu] 2 30247 cjnaz/rclonesync (https://github.com/cjnaz/rclonesync-V2) 30248 .IP \[bu] 2 30249 ConorWilliams/rsinc (https://github.com/ConorWilliams/rsinc) 30250 .IP \[bu] 2 30251 jwink3101/syncrclone (https://github.com/Jwink3101/syncrclone) 30252 .IP \[bu] 2 30253 DavideRossi/upback (https://github.com/DavideRossi/upback) 30254 .PP 30255 Bisync adopts the differential synchronization technique, which is based 30256 on keeping history of changes performed by both synchronizing sides. 30257 See the \f[I]Dual Shadow Method\f[R] section in Neil Fraser\[aq]s 30258 article (https://neil.fraser.name/writing/sync/). 30259 .PP 30260 Also note a number of academic publications by Benjamin 30261 Pierce (http://www.cis.upenn.edu/%7Ebcpierce/papers/index.shtml#File%20Synchronization) 30262 about \f[I]Unison\f[R] and synchronization in general. 30263 .SS Changelog 30264 .SS \f[C]v1.66\f[R] 30265 .IP \[bu] 2 30266 Copies and deletes are now handled in one operation instead of two 30267 .IP \[bu] 2 30268 \f[C]--track-renames\f[R] and \f[C]--backup-dir\f[R] are now supported 30269 .IP \[bu] 2 30270 Partial uploads known issue on 30271 \f[C]local\f[R]/\f[C]ftp\f[R]/\f[C]sftp\f[R] has been resolved (unless 30272 using \f[C]--inplace\f[R]) 30273 .IP \[bu] 2 30274 Final listings are now generated from sync results, to avoid needing to 30275 re-list 30276 .IP \[bu] 2 30277 Bisync is now much more resilient to changes that happen during a bisync 30278 run, and far less prone to critical errors / undetected changes 30279 .IP \[bu] 2 30280 Bisync is now capable of rolling a file listing back in cases of 30281 uncertainty, essentially marking the file as needing to be rechecked 30282 next time. 30283 .IP \[bu] 2 30284 A few basic terminal colors are now supported, controllable with 30285 \f[C]--color\f[R] (https://rclone.org/docs/#color-when) 30286 (\f[C]AUTO\f[R]|\f[C]NEVER\f[R]|\f[C]ALWAYS\f[R]) 30287 .IP \[bu] 2 30288 Initial listing snapshots of Path1 and Path2 are now generated 30289 concurrently, using the same \[dq]march\[dq] infrastructure as 30290 \f[C]check\f[R] and \f[C]sync\f[R], for performance improvements and 30291 less risk of 30292 error (https://forum.rclone.org/t/bisync-bugs-and-feature-requests/37636#:~:text=4.%20Listings%20should%20alternate%20between%20paths%20to%20minimize%20errors). 30293 .IP \[bu] 2 30294 Fixed handling of unicode normalization and case insensitivity, support 30295 for \f[C]--fix-case\f[R] (https://rclone.org/docs/#fix-case), 30296 \f[C]--ignore-case-sync\f[R], \f[C]--no-unicode-normalization\f[R] 30297 .IP \[bu] 2 30298 \f[C]--resync\f[R] is now much more efficient (especially for users of 30299 \f[C]--create-empty-src-dirs\f[R]) 30300 .IP \[bu] 2 30301 Google Docs (and other files of unknown size) are now supported (with 30302 the same options as in \f[C]sync\f[R]) 30303 .IP \[bu] 2 30304 Equality checks before a sync conflict rename now fall back to 30305 \f[C]cryptcheck\f[R] (when possible) or \f[C]--download\f[R], instead of 30306 of \f[C]--size-only\f[R], when \f[C]check\f[R] is not available. 30307 .IP \[bu] 2 30308 Bisync no longer fails to find the correct listing file when configs are 30309 overridden with backend-specific flags. 30310 .IP \[bu] 2 30311 Bisync now fully supports comparing based on any combination of size, 30312 modtime, and checksum, lifting the prior restriction on backends without 30313 modtime support. 30314 .IP \[bu] 2 30315 Bisync now supports a \[dq]Graceful Shutdown\[dq] mode to cleanly cancel 30316 a run early without requiring \f[C]--resync\f[R]. 30317 .IP \[bu] 2 30318 New \f[C]--recover\f[R] flag allows robust recovery in the event of 30319 interruptions, without requiring \f[C]--resync\f[R]. 30320 .IP \[bu] 2 30321 A new \f[C]--max-lock\f[R] setting allows lock files to automatically 30322 renew and expire, for better automatic recovery when a run is 30323 interrupted. 30324 .IP \[bu] 2 30325 Bisync now supports auto-resolving sync conflicts and customizing rename 30326 behavior with new \f[C]--conflict-resolve\f[R], 30327 \f[C]--conflict-loser\f[R], and \f[C]--conflict-suffix\f[R] flags. 30328 .IP \[bu] 2 30329 A new \f[C]--resync-mode\f[R] flag allows more control over which 30330 version of a file gets kept during a \f[C]--resync\f[R]. 30331 .IP \[bu] 2 30332 Bisync now supports 30333 \f[C]--retries\f[R] (https://rclone.org/docs/#retries-int) and 30334 \f[C]--retries-sleep\f[R] (when \f[C]--resilient\f[R] is set.) 30335 .SS \f[C]v1.64\f[R] 30336 .IP \[bu] 2 30337 Fixed an 30338 issue (https://forum.rclone.org/t/bisync-bugs-and-feature-requests/37636#:~:text=1.%20Dry%20runs%20are%20not%20completely%20dry) 30339 causing dry runs to inadvertently commit filter changes 30340 .IP \[bu] 2 30341 Fixed an 30342 issue (https://forum.rclone.org/t/bisync-bugs-and-feature-requests/37636#:~:text=2.%20%2D%2Dresync%20deletes%20data%2C%20contrary%20to%20docs) 30343 causing \f[C]--resync\f[R] to erroneously delete empty folders and 30344 duplicate files unique to Path2 30345 .IP \[bu] 2 30346 \f[C]--check-access\f[R] is now enforced during \f[C]--resync\f[R], 30347 preventing data loss in certain user error 30348 scenarios (https://forum.rclone.org/t/bisync-bugs-and-feature-requests/37636#:~:text=%2D%2Dcheck%2Daccess%20doesn%27t%20always%20fail%20when%20it%20should) 30349 .IP \[bu] 2 30350 Fixed an 30351 issue (https://forum.rclone.org/t/bisync-bugs-and-feature-requests/37636#:~:text=5.%20Bisync%20reads%20files%20in%20excluded%20directories%20during%20delete%20operations) 30352 causing bisync to consider more files than necessary due to overbroad 30353 filters during delete operations 30354 .IP \[bu] 2 30355 Improved detection of false positive change 30356 conflicts (https://forum.rclone.org/t/bisync-bugs-and-feature-requests/37636#:~:text=1.%20Identical%20files%20should%20be%20left%20alone%2C%20even%20if%20new/newer/changed%20on%20both%20sides) 30357 (identical files are now left alone instead of renamed) 30358 .IP \[bu] 2 30359 Added support for 30360 \f[C]--create-empty-src-dirs\f[R] (https://forum.rclone.org/t/bisync-bugs-and-feature-requests/37636#:~:text=3.%20Bisync%20should%20create/delete%20empty%20directories%20as%20sync%20does%2C%20when%20%2D%2Dcreate%2Dempty%2Dsrc%2Ddirs%20is%20passed) 30361 .IP \[bu] 2 30362 Added experimental \f[C]--resilient\f[R] mode to allow recovery from 30363 self-correctable 30364 errors (https://forum.rclone.org/t/bisync-bugs-and-feature-requests/37636#:~:text=2.%20Bisync%20should%20be%20more%20resilient%20to%20self%2Dcorrectable%20errors) 30365 .IP \[bu] 2 30366 Added new \f[C]--ignore-listing-checksum\f[R] 30367 flag (https://forum.rclone.org/t/bisync-bugs-and-feature-requests/37636#:~:text=6.%20%2D%2Dignore%2Dchecksum%20should%20be%20split%20into%20two%20flags%20for%20separate%20purposes) 30368 to distinguish from \f[C]--ignore-checksum\f[R] 30369 .IP \[bu] 2 30370 Performance 30371 improvements (https://forum.rclone.org/t/bisync-bugs-and-feature-requests/37636#:~:text=6.%20Deletes%20take%20several%20times%20longer%20than%20copies) 30372 for large remotes 30373 .IP \[bu] 2 30374 Documentation and testing improvements 30375 .SH Release signing 30376 .PP 30377 The hashes of the binary artefacts of the rclone release are signed with 30378 a public PGP/GPG key. 30379 This can be verified manually as described below. 30380 .PP 30381 The same mechanism is also used by rclone 30382 selfupdate (https://rclone.org/commands/rclone_selfupdate/) to verify 30383 that the release has not been tampered with before the new update is 30384 installed. 30385 This checks the SHA256 hash and the signature with a public key compiled 30386 into the rclone binary. 30387 .SS Release signing key 30388 .PP 30389 You may obtain the release signing key from: 30390 .IP \[bu] 2 30391 From KEYS on this website - this file contains all past signing keys 30392 also. 30393 .IP \[bu] 2 30394 The git repository hosted on GitHub - 30395 https://github.com/artpar/artpar/blob/master/docs/content/KEYS 30396 .IP \[bu] 2 30397 \f[C]gpg --keyserver hkps://keys.openpgp.org --search nick\[at]craig-wood.com\f[R] 30398 .IP \[bu] 2 30399 \f[C]gpg --keyserver hkps://keyserver.ubuntu.com --search nick\[at]craig-wood.com\f[R] 30400 .IP \[bu] 2 30401 https://www.craig-wood.com/nick/pub/pgp-key.txt 30402 .PP 30403 After importing the key, verify that the fingerprint of one of the keys 30404 matches: \f[C]FBF737ECE9F8AB18604BD2AC93935E02FF3B54FA\f[R] as this key 30405 is used for signing. 30406 .PP 30407 We recommend that you cross-check the fingerprint shown above through 30408 the domains listed below. 30409 By cross-checking the integrity of the fingerprint across multiple 30410 domains you can be confident that you obtained the correct key. 30411 .IP \[bu] 2 30412 The source for this page on 30413 GitHub (https://github.com/artpar/artpar/blob/master/docs/content/release_signing.md). 30414 .IP \[bu] 2 30415 Through DNS \f[C]dig key.rclone.org txt\f[R] 30416 .PP 30417 If you find anything that doesn\[aq]t not match, please contact the 30418 developers at once. 30419 .SS How to verify the release 30420 .PP 30421 In the release directory you will see the release files and some files 30422 called \f[C]MD5SUMS\f[R], \f[C]SHA1SUMS\f[R] and \f[C]SHA256SUMS\f[R]. 30423 .IP 30424 .nf 30425 \f[C] 30426 $ rclone lsf --http-url https://downloads.rclone.org/v1.63.1 :http: 30427 MD5SUMS 30428 SHA1SUMS 30429 SHA256SUMS 30430 rclone-v1.63.1-freebsd-386.zip 30431 rclone-v1.63.1-freebsd-amd64.zip 30432 \&... 30433 rclone-v1.63.1-windows-arm64.zip 30434 rclone-v1.63.1.tar.gz 30435 version.txt 30436 \f[R] 30437 .fi 30438 .PP 30439 The \f[C]MD5SUMS\f[R], \f[C]SHA1SUMS\f[R] and \f[C]SHA256SUMS\f[R] 30440 contain hashes of the binary files in the release directory along with a 30441 signature. 30442 .PP 30443 For example: 30444 .IP 30445 .nf 30446 \f[C] 30447 $ rclone cat --http-url https://downloads.rclone.org/v1.63.1 :http:SHA256SUMS 30448 -----BEGIN PGP SIGNED MESSAGE----- 30449 Hash: SHA1 30450 30451 f6d1b2d7477475ce681bdce8cb56f7870f174cb6b2a9ac5d7b3764296ea4a113 rclone-v1.63.1-freebsd-386.zip 30452 7266febec1f01a25d6575de51c44ddf749071a4950a6384e4164954dff7ac37e rclone-v1.63.1-freebsd-amd64.zip 30453 \&... 30454 66ca083757fb22198309b73879831ed2b42309892394bf193ff95c75dff69c73 rclone-v1.63.1-windows-amd64.zip 30455 bbb47c16882b6c5f2e8c1b04229378e28f68734c613321ef0ea2263760f74cd0 rclone-v1.63.1-windows-arm64.zip 30456 -----BEGIN PGP SIGNATURE----- 30457 30458 iF0EARECAB0WIQT79zfs6firGGBL0qyTk14C/ztU+gUCZLVKJQAKCRCTk14C/ztU 30459 +pZuAJ0XJ+QWLP/3jCtkmgcgc4KAwd/rrwCcCRZQ7E+oye1FPY46HOVzCFU3L7g= 30460 =8qrL 30461 -----END PGP SIGNATURE----- 30462 \f[R] 30463 .fi 30464 .SS Download the files 30465 .PP 30466 The first step is to download the binary and SUMs file and verify that 30467 the SUMs you have downloaded match. 30468 Here we download \f[C]rclone-v1.63.1-windows-amd64.zip\f[R] - choose the 30469 binary (or binaries) appropriate to your architecture. 30470 We\[aq]ve also chosen the \f[C]SHA256SUMS\f[R] as these are the most 30471 secure. 30472 You could verify the other types of hash also for extra security. 30473 \f[C]rclone selfupdate\f[R] verifies just the \f[C]SHA256SUMS\f[R]. 30474 .IP 30475 .nf 30476 \f[C] 30477 $ mkdir /tmp/check 30478 $ cd /tmp/check 30479 $ rclone copy --http-url https://downloads.rclone.org/v1.63.1 :http:SHA256SUMS . 30480 $ rclone copy --http-url https://downloads.rclone.org/v1.63.1 :http:rclone-v1.63.1-windows-amd64.zip . 30481 \f[R] 30482 .fi 30483 .SS Verify the signatures 30484 .PP 30485 First verify the signatures on the SHA256 file. 30486 .PP 30487 Import the key. 30488 See above for ways to verify this key is correct. 30489 .IP 30490 .nf 30491 \f[C] 30492 $ gpg --keyserver keyserver.ubuntu.com --receive-keys FBF737ECE9F8AB18604BD2AC93935E02FF3B54FA 30493 gpg: key 93935E02FF3B54FA: public key \[dq]Nick Craig-Wood <nick\[at]craig-wood.com>\[dq] imported 30494 gpg: Total number processed: 1 30495 gpg: imported: 1 30496 \f[R] 30497 .fi 30498 .PP 30499 Then check the signature: 30500 .IP 30501 .nf 30502 \f[C] 30503 $ gpg --verify SHA256SUMS 30504 gpg: Signature made Mon 17 Jul 2023 15:03:17 BST 30505 gpg: using DSA key FBF737ECE9F8AB18604BD2AC93935E02FF3B54FA 30506 gpg: Good signature from \[dq]Nick Craig-Wood <nick\[at]craig-wood.com>\[dq] [ultimate] 30507 \f[R] 30508 .fi 30509 .PP 30510 Verify the signature was good and is using the fingerprint shown above. 30511 .PP 30512 Repeat for \f[C]MD5SUMS\f[R] and \f[C]SHA1SUMS\f[R] if desired. 30513 .SS Verify the hashes 30514 .PP 30515 Now that we know the signatures on the hashes are OK we can verify the 30516 binaries match the hashes, completing the verification. 30517 .IP 30518 .nf 30519 \f[C] 30520 $ sha256sum -c SHA256SUMS 2>&1 | grep OK 30521 rclone-v1.63.1-windows-amd64.zip: OK 30522 \f[R] 30523 .fi 30524 .PP 30525 Or do the check with rclone 30526 .IP 30527 .nf 30528 \f[C] 30529 $ rclone hashsum sha256 -C SHA256SUMS rclone-v1.63.1-windows-amd64.zip 30530 2023/09/11 10:53:58 NOTICE: SHA256SUMS: improperly formatted checksum line 0 30531 2023/09/11 10:53:58 NOTICE: SHA256SUMS: improperly formatted checksum line 1 30532 2023/09/11 10:53:58 NOTICE: SHA256SUMS: improperly formatted checksum line 49 30533 2023/09/11 10:53:58 NOTICE: SHA256SUMS: 4 warning(s) suppressed... 30534 = rclone-v1.63.1-windows-amd64.zip 30535 2023/09/11 10:53:58 NOTICE: Local file system at /tmp/check: 0 differences found 30536 2023/09/11 10:53:58 NOTICE: Local file system at /tmp/check: 1 matching files 30537 \f[R] 30538 .fi 30539 .SS Verify signatures and hashes together 30540 .PP 30541 You can verify the signatures and hashes in one command line like this: 30542 .IP 30543 .nf 30544 \f[C] 30545 $ gpg --decrypt SHA256SUMS | sha256sum -c --ignore-missing 30546 gpg: Signature made Mon 17 Jul 2023 15:03:17 BST 30547 gpg: using DSA key FBF737ECE9F8AB18604BD2AC93935E02FF3B54FA 30548 gpg: Good signature from \[dq]Nick Craig-Wood <nick\[at]craig-wood.com>\[dq] [ultimate] 30549 gpg: aka \[dq]Nick Craig-Wood <nick\[at]memset.com>\[dq] [unknown] 30550 rclone-v1.63.1-windows-amd64.zip: OK 30551 \f[R] 30552 .fi 30553 .SH 1Fichier 30554 .PP 30555 This is a backend for the 1fichier (https://1fichier.com) cloud storage 30556 service. 30557 Note that a Premium subscription is required to use the API. 30558 .PP 30559 Paths are specified as \f[C]remote:path\f[R] 30560 .PP 30561 Paths may be as deep as required, e.g. 30562 \f[C]remote:directory/subdirectory\f[R]. 30563 .SS Configuration 30564 .PP 30565 The initial setup for 1Fichier involves getting the API key from the 30566 website which you need to do in your browser. 30567 .PP 30568 Here is an example of how to make a remote called \f[C]remote\f[R]. 30569 First run: 30570 .IP 30571 .nf 30572 \f[C] 30573 rclone config 30574 \f[R] 30575 .fi 30576 .PP 30577 This will guide you through an interactive setup process: 30578 .IP 30579 .nf 30580 \f[C] 30581 No remotes found, make a new one? 30582 n) New remote 30583 s) Set configuration password 30584 q) Quit config 30585 n/s/q> n 30586 name> remote 30587 Type of storage to configure. 30588 Enter a string value. Press Enter for the default (\[dq]\[dq]). 30589 Choose a number from below, or type in your own value 30590 [snip] 30591 XX / 1Fichier 30592 \[rs] \[dq]fichier\[dq] 30593 [snip] 30594 Storage> fichier 30595 ** See help for fichier backend at: https://rclone.org/fichier/ ** 30596 30597 Your API Key, get it from https://1fichier.com/console/params.pl 30598 Enter a string value. Press Enter for the default (\[dq]\[dq]). 30599 api_key> example_key 30600 30601 Edit advanced config? (y/n) 30602 y) Yes 30603 n) No 30604 y/n> 30605 Remote config 30606 -------------------- 30607 [remote] 30608 type = fichier 30609 api_key = example_key 30610 -------------------- 30611 y) Yes this is OK 30612 e) Edit this remote 30613 d) Delete this remote 30614 y/e/d> y 30615 \f[R] 30616 .fi 30617 .PP 30618 Once configured you can then use \f[C]rclone\f[R] like this, 30619 .PP 30620 List directories in top level of your 1Fichier account 30621 .IP 30622 .nf 30623 \f[C] 30624 rclone lsd remote: 30625 \f[R] 30626 .fi 30627 .PP 30628 List all the files in your 1Fichier account 30629 .IP 30630 .nf 30631 \f[C] 30632 rclone ls remote: 30633 \f[R] 30634 .fi 30635 .PP 30636 To copy a local directory to a 1Fichier directory called backup 30637 .IP 30638 .nf 30639 \f[C] 30640 rclone copy /home/source remote:backup 30641 \f[R] 30642 .fi 30643 .SS Modification times and hashes 30644 .PP 30645 1Fichier does not support modification times. 30646 It supports the Whirlpool hash algorithm. 30647 .SS Duplicated files 30648 .PP 30649 1Fichier can have two files with exactly the same name and path (unlike 30650 a normal file system). 30651 .PP 30652 Duplicated files cause problems with the syncing and you will see 30653 messages in the log about duplicates. 30654 .SS Restricted filename characters 30655 .PP 30656 In addition to the default restricted characters 30657 set (https://rclone.org/overview/#restricted-characters) the following 30658 characters are also replaced: 30659 .PP 30660 .TS 30661 tab(@); 30662 l c c. 30663 T{ 30664 Character 30665 T}@T{ 30666 Value 30667 T}@T{ 30668 Replacement 30669 T} 30670 _ 30671 T{ 30672 \[rs] 30673 T}@T{ 30674 0x5C 30675 T}@T{ 30676 \[uFF3C] 30677 T} 30678 T{ 30679 < 30680 T}@T{ 30681 0x3C 30682 T}@T{ 30683 \[uFF1C] 30684 T} 30685 T{ 30686 > 30687 T}@T{ 30688 0x3E 30689 T}@T{ 30690 \[uFF1E] 30691 T} 30692 T{ 30693 \[dq] 30694 T}@T{ 30695 0x22 30696 T}@T{ 30697 \[uFF02] 30698 T} 30699 T{ 30700 $ 30701 T}@T{ 30702 0x24 30703 T}@T{ 30704 \[uFF04] 30705 T} 30706 T{ 30707 \[ga] 30708 T}@T{ 30709 0x60 30710 T}@T{ 30711 \[uFF40] 30712 T} 30713 T{ 30714 \[aq] 30715 T}@T{ 30716 0x27 30717 T}@T{ 30718 \[uFF07] 30719 T} 30720 .TE 30721 .PP 30722 File names can also not start or end with the following characters. 30723 These only get replaced if they are the first or last character in the 30724 name: 30725 .PP 30726 .TS 30727 tab(@); 30728 l c c. 30729 T{ 30730 Character 30731 T}@T{ 30732 Value 30733 T}@T{ 30734 Replacement 30735 T} 30736 _ 30737 T{ 30738 SP 30739 T}@T{ 30740 0x20 30741 T}@T{ 30742 \[u2420] 30743 T} 30744 .TE 30745 .PP 30746 Invalid UTF-8 bytes will also be 30747 replaced (https://rclone.org/overview/#invalid-utf8), as they can\[aq]t 30748 be used in JSON strings. 30749 .SS Standard options 30750 .PP 30751 Here are the Standard options specific to fichier (1Fichier). 30752 .SS --fichier-api-key 30753 .PP 30754 Your API Key, get it from https://1fichier.com/console/params.pl. 30755 .PP 30756 Properties: 30757 .IP \[bu] 2 30758 Config: api_key 30759 .IP \[bu] 2 30760 Env Var: RCLONE_FICHIER_API_KEY 30761 .IP \[bu] 2 30762 Type: string 30763 .IP \[bu] 2 30764 Required: false 30765 .SS Advanced options 30766 .PP 30767 Here are the Advanced options specific to fichier (1Fichier). 30768 .SS --fichier-shared-folder 30769 .PP 30770 If you want to download a shared folder, add this parameter. 30771 .PP 30772 Properties: 30773 .IP \[bu] 2 30774 Config: shared_folder 30775 .IP \[bu] 2 30776 Env Var: RCLONE_FICHIER_SHARED_FOLDER 30777 .IP \[bu] 2 30778 Type: string 30779 .IP \[bu] 2 30780 Required: false 30781 .SS --fichier-file-password 30782 .PP 30783 If you want to download a shared file that is password protected, add 30784 this parameter. 30785 .PP 30786 \f[B]NB\f[R] Input to this must be obscured - see rclone 30787 obscure (https://rclone.org/commands/rclone_obscure/). 30788 .PP 30789 Properties: 30790 .IP \[bu] 2 30791 Config: file_password 30792 .IP \[bu] 2 30793 Env Var: RCLONE_FICHIER_FILE_PASSWORD 30794 .IP \[bu] 2 30795 Type: string 30796 .IP \[bu] 2 30797 Required: false 30798 .SS --fichier-folder-password 30799 .PP 30800 If you want to list the files in a shared folder that is password 30801 protected, add this parameter. 30802 .PP 30803 \f[B]NB\f[R] Input to this must be obscured - see rclone 30804 obscure (https://rclone.org/commands/rclone_obscure/). 30805 .PP 30806 Properties: 30807 .IP \[bu] 2 30808 Config: folder_password 30809 .IP \[bu] 2 30810 Env Var: RCLONE_FICHIER_FOLDER_PASSWORD 30811 .IP \[bu] 2 30812 Type: string 30813 .IP \[bu] 2 30814 Required: false 30815 .SS --fichier-cdn 30816 .PP 30817 Set if you wish to use CDN download links. 30818 .PP 30819 Properties: 30820 .IP \[bu] 2 30821 Config: cdn 30822 .IP \[bu] 2 30823 Env Var: RCLONE_FICHIER_CDN 30824 .IP \[bu] 2 30825 Type: bool 30826 .IP \[bu] 2 30827 Default: false 30828 .SS --fichier-encoding 30829 .PP 30830 The encoding for the backend. 30831 .PP 30832 See the encoding section in the 30833 overview (https://rclone.org/overview/#encoding) for more info. 30834 .PP 30835 Properties: 30836 .IP \[bu] 2 30837 Config: encoding 30838 .IP \[bu] 2 30839 Env Var: RCLONE_FICHIER_ENCODING 30840 .IP \[bu] 2 30841 Type: Encoding 30842 .IP \[bu] 2 30843 Default: 30844 Slash,LtGt,DoubleQuote,SingleQuote,BackQuote,Dollar,BackSlash,Del,Ctl,LeftSpace,RightSpace,InvalidUtf8,Dot 30845 .SS --fichier-description 30846 .PP 30847 Description of the remote 30848 .PP 30849 Properties: 30850 .IP \[bu] 2 30851 Config: description 30852 .IP \[bu] 2 30853 Env Var: RCLONE_FICHIER_DESCRIPTION 30854 .IP \[bu] 2 30855 Type: string 30856 .IP \[bu] 2 30857 Required: false 30858 .SS Limitations 30859 .PP 30860 \f[C]rclone about\f[R] is not supported by the 1Fichier backend. 30861 Backends without this capability cannot determine free space for an 30862 rclone mount or use policy \f[C]mfs\f[R] (most free space) as a member 30863 of an rclone union remote. 30864 .PP 30865 See List of backends that do not support rclone 30866 about (https://rclone.org/overview/#optional-features) and rclone 30867 about (https://rclone.org/commands/rclone_about/) 30868 .SH Alias 30869 .PP 30870 The \f[C]alias\f[R] remote provides a new name for another remote. 30871 .PP 30872 Paths may be as deep as required or a local path, e.g. 30873 \f[C]remote:directory/subdirectory\f[R] or 30874 \f[C]/directory/subdirectory\f[R]. 30875 .PP 30876 During the initial setup with \f[C]rclone config\f[R] you will specify 30877 the target remote. 30878 The target remote can either be a local path or another remote. 30879 .PP 30880 Subfolders can be used in target remote. 30881 Assume an alias remote named \f[C]backup\f[R] with the target 30882 \f[C]mydrive:private/backup\f[R]. 30883 Invoking \f[C]rclone mkdir backup:desktop\f[R] is exactly the same as 30884 invoking \f[C]rclone mkdir mydrive:private/backup/desktop\f[R]. 30885 .PP 30886 There will be no special handling of paths containing \f[C]..\f[R] 30887 segments. 30888 Invoking \f[C]rclone mkdir backup:../desktop\f[R] is exactly the same as 30889 invoking \f[C]rclone mkdir mydrive:private/backup/../desktop\f[R]. 30890 The empty path is not allowed as a remote. 30891 To alias the current directory use \f[C].\f[R] instead. 30892 .PP 30893 The target remote can also be a connection 30894 string (https://rclone.org/docs/#connection-strings). 30895 This can be used to modify the config of a remote for different uses, 30896 e.g. 30897 the alias \f[C]myDriveTrash\f[R] with the target remote 30898 \f[C]myDrive,trashed_only:\f[R] can be used to only show the trashed 30899 files in \f[C]myDrive\f[R]. 30900 .SS Configuration 30901 .PP 30902 Here is an example of how to make an alias called \f[C]remote\f[R] for 30903 local folder. 30904 First run: 30905 .IP 30906 .nf 30907 \f[C] 30908 rclone config 30909 \f[R] 30910 .fi 30911 .PP 30912 This will guide you through an interactive setup process: 30913 .IP 30914 .nf 30915 \f[C] 30916 No remotes found, make a new one? 30917 n) New remote 30918 s) Set configuration password 30919 q) Quit config 30920 n/s/q> n 30921 name> remote 30922 Type of storage to configure. 30923 Choose a number from below, or type in your own value 30924 [snip] 30925 XX / Alias for an existing remote 30926 \[rs] \[dq]alias\[dq] 30927 [snip] 30928 Storage> alias 30929 Remote or path to alias. 30930 Can be \[dq]myremote:path/to/dir\[dq], \[dq]myremote:bucket\[dq], \[dq]myremote:\[dq] or \[dq]/local/path\[dq]. 30931 remote> /mnt/storage/backup 30932 Remote config 30933 -------------------- 30934 [remote] 30935 remote = /mnt/storage/backup 30936 -------------------- 30937 y) Yes this is OK 30938 e) Edit this remote 30939 d) Delete this remote 30940 y/e/d> y 30941 Current remotes: 30942 30943 Name Type 30944 ==== ==== 30945 remote alias 30946 30947 e) Edit existing remote 30948 n) New remote 30949 d) Delete remote 30950 r) Rename remote 30951 c) Copy remote 30952 s) Set configuration password 30953 q) Quit config 30954 e/n/d/r/c/s/q> q 30955 \f[R] 30956 .fi 30957 .PP 30958 Once configured you can then use \f[C]rclone\f[R] like this, 30959 .PP 30960 List directories in top level in \f[C]/mnt/storage/backup\f[R] 30961 .IP 30962 .nf 30963 \f[C] 30964 rclone lsd remote: 30965 \f[R] 30966 .fi 30967 .PP 30968 List all the files in \f[C]/mnt/storage/backup\f[R] 30969 .IP 30970 .nf 30971 \f[C] 30972 rclone ls remote: 30973 \f[R] 30974 .fi 30975 .PP 30976 Copy another local directory to the alias directory called source 30977 .IP 30978 .nf 30979 \f[C] 30980 rclone copy /home/source remote:source 30981 \f[R] 30982 .fi 30983 .SS Standard options 30984 .PP 30985 Here are the Standard options specific to alias (Alias for an existing 30986 remote). 30987 .SS --alias-remote 30988 .PP 30989 Remote or path to alias. 30990 .PP 30991 Can be \[dq]myremote:path/to/dir\[dq], \[dq]myremote:bucket\[dq], 30992 \[dq]myremote:\[dq] or \[dq]/local/path\[dq]. 30993 .PP 30994 Properties: 30995 .IP \[bu] 2 30996 Config: remote 30997 .IP \[bu] 2 30998 Env Var: RCLONE_ALIAS_REMOTE 30999 .IP \[bu] 2 31000 Type: string 31001 .IP \[bu] 2 31002 Required: true 31003 .SS Advanced options 31004 .PP 31005 Here are the Advanced options specific to alias (Alias for an existing 31006 remote). 31007 .SS --alias-description 31008 .PP 31009 Description of the remote 31010 .PP 31011 Properties: 31012 .IP \[bu] 2 31013 Config: description 31014 .IP \[bu] 2 31015 Env Var: RCLONE_ALIAS_DESCRIPTION 31016 .IP \[bu] 2 31017 Type: string 31018 .IP \[bu] 2 31019 Required: false 31020 .SH Amazon S3 Storage Providers 31021 .PP 31022 The S3 backend can be used with a number of different providers: 31023 .IP \[bu] 2 31024 AWS S3 31025 .IP \[bu] 2 31026 Alibaba Cloud (Aliyun) Object Storage System (OSS) 31027 .IP \[bu] 2 31028 Ceph 31029 .IP \[bu] 2 31030 China Mobile Ecloud Elastic Object Storage (EOS) 31031 .IP \[bu] 2 31032 Cloudflare R2 31033 .IP \[bu] 2 31034 Arvan Cloud Object Storage (AOS) 31035 .IP \[bu] 2 31036 DigitalOcean Spaces 31037 .IP \[bu] 2 31038 Dreamhost 31039 .IP \[bu] 2 31040 GCS 31041 .IP \[bu] 2 31042 Huawei OBS 31043 .IP \[bu] 2 31044 IBM COS S3 31045 .IP \[bu] 2 31046 IDrive e2 31047 .IP \[bu] 2 31048 IONOS Cloud 31049 .IP \[bu] 2 31050 Leviia Object Storage 31051 .IP \[bu] 2 31052 Liara Object Storage 31053 .IP \[bu] 2 31054 Linode Object Storage 31055 .IP \[bu] 2 31056 Minio 31057 .IP \[bu] 2 31058 Petabox 31059 .IP \[bu] 2 31060 Qiniu Cloud Object Storage (Kodo) 31061 .IP \[bu] 2 31062 RackCorp Object Storage 31063 .IP \[bu] 2 31064 Rclone Serve S3 31065 .IP \[bu] 2 31066 Scaleway 31067 .IP \[bu] 2 31068 Seagate Lyve Cloud 31069 .IP \[bu] 2 31070 SeaweedFS 31071 .IP \[bu] 2 31072 StackPath 31073 .IP \[bu] 2 31074 Storj 31075 .IP \[bu] 2 31076 Synology C2 Object Storage 31077 .IP \[bu] 2 31078 Tencent Cloud Object Storage (COS) 31079 .IP \[bu] 2 31080 Wasabi 31081 .PP 31082 Paths are specified as \f[C]remote:bucket\f[R] (or \f[C]remote:\f[R] for 31083 the \f[C]lsd\f[R] command.) You may put subdirectories in too, e.g. 31084 \f[C]remote:bucket/path/to/dir\f[R]. 31085 .PP 31086 Once you have made a remote (see the provider specific section above) 31087 you can use it like this: 31088 .PP 31089 See all buckets 31090 .IP 31091 .nf 31092 \f[C] 31093 rclone lsd remote: 31094 \f[R] 31095 .fi 31096 .PP 31097 Make a new bucket 31098 .IP 31099 .nf 31100 \f[C] 31101 rclone mkdir remote:bucket 31102 \f[R] 31103 .fi 31104 .PP 31105 List the contents of a bucket 31106 .IP 31107 .nf 31108 \f[C] 31109 rclone ls remote:bucket 31110 \f[R] 31111 .fi 31112 .PP 31113 Sync \f[C]/home/local/directory\f[R] to the remote bucket, deleting any 31114 excess files in the bucket. 31115 .IP 31116 .nf 31117 \f[C] 31118 rclone sync --interactive /home/local/directory remote:bucket 31119 \f[R] 31120 .fi 31121 .SS Configuration 31122 .PP 31123 Here is an example of making an s3 configuration for the AWS S3 31124 provider. 31125 Most applies to the other providers as well, any differences are 31126 described below. 31127 .PP 31128 First run 31129 .IP 31130 .nf 31131 \f[C] 31132 rclone config 31133 \f[R] 31134 .fi 31135 .PP 31136 This will guide you through an interactive setup process. 31137 .IP 31138 .nf 31139 \f[C] 31140 No remotes found, make a new one? 31141 n) New remote 31142 s) Set configuration password 31143 q) Quit config 31144 n/s/q> n 31145 name> remote 31146 Type of storage to configure. 31147 Choose a number from below, or type in your own value 31148 [snip] 31149 XX / Amazon S3 Compliant Storage Providers including AWS, ... 31150 \[rs] \[dq]s3\[dq] 31151 [snip] 31152 Storage> s3 31153 Choose your S3 provider. 31154 Choose a number from below, or type in your own value 31155 1 / Amazon Web Services (AWS) S3 31156 \[rs] \[dq]AWS\[dq] 31157 2 / Ceph Object Storage 31158 \[rs] \[dq]Ceph\[dq] 31159 3 / DigitalOcean Spaces 31160 \[rs] \[dq]DigitalOcean\[dq] 31161 4 / Dreamhost DreamObjects 31162 \[rs] \[dq]Dreamhost\[dq] 31163 5 / IBM COS S3 31164 \[rs] \[dq]IBMCOS\[dq] 31165 6 / Minio Object Storage 31166 \[rs] \[dq]Minio\[dq] 31167 7 / Wasabi Object Storage 31168 \[rs] \[dq]Wasabi\[dq] 31169 8 / Any other S3 compatible provider 31170 \[rs] \[dq]Other\[dq] 31171 provider> 1 31172 Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars). Only applies if access_key_id and secret_access_key is blank. 31173 Choose a number from below, or type in your own value 31174 1 / Enter AWS credentials in the next step 31175 \[rs] \[dq]false\[dq] 31176 2 / Get AWS credentials from the environment (env vars or IAM) 31177 \[rs] \[dq]true\[dq] 31178 env_auth> 1 31179 AWS Access Key ID - leave blank for anonymous access or runtime credentials. 31180 access_key_id> XXX 31181 AWS Secret Access Key (password) - leave blank for anonymous access or runtime credentials. 31182 secret_access_key> YYY 31183 Region to connect to. 31184 Choose a number from below, or type in your own value 31185 / The default endpoint - a good choice if you are unsure. 31186 1 | US Region, Northern Virginia, or Pacific Northwest. 31187 | Leave location constraint empty. 31188 \[rs] \[dq]us-east-1\[dq] 31189 / US East (Ohio) Region 31190 2 | Needs location constraint us-east-2. 31191 \[rs] \[dq]us-east-2\[dq] 31192 / US West (Oregon) Region 31193 3 | Needs location constraint us-west-2. 31194 \[rs] \[dq]us-west-2\[dq] 31195 / US West (Northern California) Region 31196 4 | Needs location constraint us-west-1. 31197 \[rs] \[dq]us-west-1\[dq] 31198 / Canada (Central) Region 31199 5 | Needs location constraint ca-central-1. 31200 \[rs] \[dq]ca-central-1\[dq] 31201 / EU (Ireland) Region 31202 6 | Needs location constraint EU or eu-west-1. 31203 \[rs] \[dq]eu-west-1\[dq] 31204 / EU (London) Region 31205 7 | Needs location constraint eu-west-2. 31206 \[rs] \[dq]eu-west-2\[dq] 31207 / EU (Frankfurt) Region 31208 8 | Needs location constraint eu-central-1. 31209 \[rs] \[dq]eu-central-1\[dq] 31210 / Asia Pacific (Singapore) Region 31211 9 | Needs location constraint ap-southeast-1. 31212 \[rs] \[dq]ap-southeast-1\[dq] 31213 / Asia Pacific (Sydney) Region 31214 10 | Needs location constraint ap-southeast-2. 31215 \[rs] \[dq]ap-southeast-2\[dq] 31216 / Asia Pacific (Tokyo) Region 31217 11 | Needs location constraint ap-northeast-1. 31218 \[rs] \[dq]ap-northeast-1\[dq] 31219 / Asia Pacific (Seoul) 31220 12 | Needs location constraint ap-northeast-2. 31221 \[rs] \[dq]ap-northeast-2\[dq] 31222 / Asia Pacific (Mumbai) 31223 13 | Needs location constraint ap-south-1. 31224 \[rs] \[dq]ap-south-1\[dq] 31225 / Asia Pacific (Hong Kong) Region 31226 14 | Needs location constraint ap-east-1. 31227 \[rs] \[dq]ap-east-1\[dq] 31228 / South America (Sao Paulo) Region 31229 15 | Needs location constraint sa-east-1. 31230 \[rs] \[dq]sa-east-1\[dq] 31231 region> 1 31232 Endpoint for S3 API. 31233 Leave blank if using AWS to use the default endpoint for the region. 31234 endpoint> 31235 Location constraint - must be set to match the Region. Used when creating buckets only. 31236 Choose a number from below, or type in your own value 31237 1 / Empty for US Region, Northern Virginia, or Pacific Northwest. 31238 \[rs] \[dq]\[dq] 31239 2 / US East (Ohio) Region. 31240 \[rs] \[dq]us-east-2\[dq] 31241 3 / US West (Oregon) Region. 31242 \[rs] \[dq]us-west-2\[dq] 31243 4 / US West (Northern California) Region. 31244 \[rs] \[dq]us-west-1\[dq] 31245 5 / Canada (Central) Region. 31246 \[rs] \[dq]ca-central-1\[dq] 31247 6 / EU (Ireland) Region. 31248 \[rs] \[dq]eu-west-1\[dq] 31249 7 / EU (London) Region. 31250 \[rs] \[dq]eu-west-2\[dq] 31251 8 / EU Region. 31252 \[rs] \[dq]EU\[dq] 31253 9 / Asia Pacific (Singapore) Region. 31254 \[rs] \[dq]ap-southeast-1\[dq] 31255 10 / Asia Pacific (Sydney) Region. 31256 \[rs] \[dq]ap-southeast-2\[dq] 31257 11 / Asia Pacific (Tokyo) Region. 31258 \[rs] \[dq]ap-northeast-1\[dq] 31259 12 / Asia Pacific (Seoul) 31260 \[rs] \[dq]ap-northeast-2\[dq] 31261 13 / Asia Pacific (Mumbai) 31262 \[rs] \[dq]ap-south-1\[dq] 31263 14 / Asia Pacific (Hong Kong) 31264 \[rs] \[dq]ap-east-1\[dq] 31265 15 / South America (Sao Paulo) Region. 31266 \[rs] \[dq]sa-east-1\[dq] 31267 location_constraint> 1 31268 Canned ACL used when creating buckets and/or storing objects in S3. 31269 For more info visit https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl 31270 Choose a number from below, or type in your own value 31271 1 / Owner gets FULL_CONTROL. No one else has access rights (default). 31272 \[rs] \[dq]private\[dq] 31273 2 / Owner gets FULL_CONTROL. The AllUsers group gets READ access. 31274 \[rs] \[dq]public-read\[dq] 31275 / Owner gets FULL_CONTROL. The AllUsers group gets READ and WRITE access. 31276 3 | Granting this on a bucket is generally not recommended. 31277 \[rs] \[dq]public-read-write\[dq] 31278 4 / Owner gets FULL_CONTROL. The AuthenticatedUsers group gets READ access. 31279 \[rs] \[dq]authenticated-read\[dq] 31280 / Object owner gets FULL_CONTROL. Bucket owner gets READ access. 31281 5 | If you specify this canned ACL when creating a bucket, Amazon S3 ignores it. 31282 \[rs] \[dq]bucket-owner-read\[dq] 31283 / Both the object owner and the bucket owner get FULL_CONTROL over the object. 31284 6 | If you specify this canned ACL when creating a bucket, Amazon S3 ignores it. 31285 \[rs] \[dq]bucket-owner-full-control\[dq] 31286 acl> 1 31287 The server-side encryption algorithm used when storing this object in S3. 31288 Choose a number from below, or type in your own value 31289 1 / None 31290 \[rs] \[dq]\[dq] 31291 2 / AES256 31292 \[rs] \[dq]AES256\[dq] 31293 server_side_encryption> 1 31294 The storage class to use when storing objects in S3. 31295 Choose a number from below, or type in your own value 31296 1 / Default 31297 \[rs] \[dq]\[dq] 31298 2 / Standard storage class 31299 \[rs] \[dq]STANDARD\[dq] 31300 3 / Reduced redundancy storage class 31301 \[rs] \[dq]REDUCED_REDUNDANCY\[dq] 31302 4 / Standard Infrequent Access storage class 31303 \[rs] \[dq]STANDARD_IA\[dq] 31304 5 / One Zone Infrequent Access storage class 31305 \[rs] \[dq]ONEZONE_IA\[dq] 31306 6 / Glacier storage class 31307 \[rs] \[dq]GLACIER\[dq] 31308 7 / Glacier Deep Archive storage class 31309 \[rs] \[dq]DEEP_ARCHIVE\[dq] 31310 8 / Intelligent-Tiering storage class 31311 \[rs] \[dq]INTELLIGENT_TIERING\[dq] 31312 9 / Glacier Instant Retrieval storage class 31313 \[rs] \[dq]GLACIER_IR\[dq] 31314 storage_class> 1 31315 Remote config 31316 -------------------- 31317 [remote] 31318 type = s3 31319 provider = AWS 31320 env_auth = false 31321 access_key_id = XXX 31322 secret_access_key = YYY 31323 region = us-east-1 31324 endpoint = 31325 location_constraint = 31326 acl = private 31327 server_side_encryption = 31328 storage_class = 31329 -------------------- 31330 y) Yes this is OK 31331 e) Edit this remote 31332 d) Delete this remote 31333 y/e/d> 31334 \f[R] 31335 .fi 31336 .SS Modification times and hashes 31337 .SS Modification times 31338 .PP 31339 The modified time is stored as metadata on the object as 31340 \f[C]X-Amz-Meta-Mtime\f[R] as floating point since the epoch, accurate 31341 to 1 ns. 31342 .PP 31343 If the modification time needs to be updated rclone will attempt to 31344 perform a server side copy to update the modification if the object can 31345 be copied in a single part. 31346 In the case the object is larger than 5Gb or is in Glacier or Glacier 31347 Deep Archive storage the object will be uploaded rather than copied. 31348 .PP 31349 Note that reading this from the object takes an additional 31350 \f[C]HEAD\f[R] request as the metadata isn\[aq]t returned in object 31351 listings. 31352 .SS Hashes 31353 .PP 31354 For small objects which weren\[aq]t uploaded as multipart uploads 31355 (objects sized below \f[C]--s3-upload-cutoff\f[R] if uploaded with 31356 rclone) rclone uses the \f[C]ETag:\f[R] header as an MD5 checksum. 31357 .PP 31358 However for objects which were uploaded as multipart uploads or with 31359 server side encryption (SSE-AWS or SSE-C) the \f[C]ETag\f[R] header is 31360 no longer the MD5 sum of the data, so rclone adds an additional piece of 31361 metadata \f[C]X-Amz-Meta-Md5chksum\f[R] which is a base64 encoded MD5 31362 hash (in the same format as is required for \f[C]Content-MD5\f[R]). 31363 You can use base64 -d and hexdump to check this value manually: 31364 .IP 31365 .nf 31366 \f[C] 31367 echo \[aq]VWTGdNx3LyXQDfA0e2Edxw==\[aq] | base64 -d | hexdump 31368 \f[R] 31369 .fi 31370 .PP 31371 or you can use \f[C]rclone check\f[R] to verify the hashes are OK. 31372 .PP 31373 For large objects, calculating this hash can take some time so the 31374 addition of this hash can be disabled with 31375 \f[C]--s3-disable-checksum\f[R]. 31376 This will mean that these objects do not have an MD5 checksum. 31377 .PP 31378 Note that reading this from the object takes an additional 31379 \f[C]HEAD\f[R] request as the metadata isn\[aq]t returned in object 31380 listings. 31381 .SS Reducing costs 31382 .SS Avoiding HEAD requests to read the modification time 31383 .PP 31384 By default, rclone will use the modification time of objects stored in 31385 S3 for syncing. 31386 This is stored in object metadata which unfortunately takes an extra 31387 HEAD request to read which can be expensive (in time and money). 31388 .PP 31389 The modification time is used by default for all operations that require 31390 checking the time a file was last updated. 31391 It allows rclone to treat the remote more like a true filesystem, but it 31392 is inefficient on S3 because it requires an extra API call to retrieve 31393 the metadata. 31394 .PP 31395 The extra API calls can be avoided when syncing (using 31396 \f[C]rclone sync\f[R] or \f[C]rclone copy\f[R]) in a few different ways, 31397 each with its own tradeoffs. 31398 .IP \[bu] 2 31399 \f[C]--size-only\f[R] 31400 .RS 2 31401 .IP \[bu] 2 31402 Only checks the size of files. 31403 .IP \[bu] 2 31404 Uses no extra transactions. 31405 .IP \[bu] 2 31406 If the file doesn\[aq]t change size then rclone won\[aq]t detect it has 31407 changed. 31408 .IP \[bu] 2 31409 \f[C]rclone sync --size-only /path/to/source s3:bucket\f[R] 31410 .RE 31411 .IP \[bu] 2 31412 \f[C]--checksum\f[R] 31413 .RS 2 31414 .IP \[bu] 2 31415 Checks the size and MD5 checksum of files. 31416 .IP \[bu] 2 31417 Uses no extra transactions. 31418 .IP \[bu] 2 31419 The most accurate detection of changes possible. 31420 .IP \[bu] 2 31421 Will cause the source to read an MD5 checksum which, if it is a local 31422 disk, will cause lots of disk activity. 31423 .IP \[bu] 2 31424 If the source and destination are both S3 this is the 31425 \f[B]recommended\f[R] flag to use for maximum efficiency. 31426 .IP \[bu] 2 31427 \f[C]rclone sync --checksum /path/to/source s3:bucket\f[R] 31428 .RE 31429 .IP \[bu] 2 31430 \f[C]--update --use-server-modtime\f[R] 31431 .RS 2 31432 .IP \[bu] 2 31433 Uses no extra transactions. 31434 .IP \[bu] 2 31435 Modification time becomes the time the object was uploaded. 31436 .IP \[bu] 2 31437 For many operations this is sufficient to determine if it needs 31438 uploading. 31439 .IP \[bu] 2 31440 Using \f[C]--update\f[R] along with \f[C]--use-server-modtime\f[R], 31441 avoids the extra API call and uploads files whose local modification 31442 time is newer than the time it was last uploaded. 31443 .IP \[bu] 2 31444 Files created with timestamps in the past will be missed by the sync. 31445 .IP \[bu] 2 31446 \f[C]rclone sync --update --use-server-modtime /path/to/source s3:bucket\f[R] 31447 .RE 31448 .PP 31449 These flags can and should be used in combination with 31450 \f[C]--fast-list\f[R] - see below. 31451 .PP 31452 If using \f[C]rclone mount\f[R] or any command using the VFS (eg 31453 \f[C]rclone serve\f[R]) commands then you might want to consider using 31454 the VFS flag \f[C]--no-modtime\f[R] which will stop rclone reading the 31455 modification time for every object. 31456 You could also use \f[C]--use-server-modtime\f[R] if you are happy with 31457 the modification times of the objects being the time of upload. 31458 .SS Avoiding GET requests to read directory listings 31459 .PP 31460 Rclone\[aq]s default directory traversal is to process each directory 31461 individually. 31462 This takes one API call per directory. 31463 Using the \f[C]--fast-list\f[R] flag will read all info about the 31464 objects into memory first using a smaller number of API calls (one per 31465 1000 objects). 31466 See the rclone docs (https://rclone.org/docs/#fast-list) for more 31467 details. 31468 .IP 31469 .nf 31470 \f[C] 31471 rclone sync --fast-list --checksum /path/to/source s3:bucket 31472 \f[R] 31473 .fi 31474 .PP 31475 \f[C]--fast-list\f[R] trades off API transactions for memory use. 31476 As a rough guide rclone uses 1k of memory per object stored, so using 31477 \f[C]--fast-list\f[R] on a sync of a million objects will use roughly 1 31478 GiB of RAM. 31479 .PP 31480 If you are only copying a small number of files into a big repository 31481 then using \f[C]--no-traverse\f[R] is a good idea. 31482 This finds objects directly instead of through directory listings. 31483 You can do a \[dq]top-up\[dq] sync very cheaply by using 31484 \f[C]--max-age\f[R] and \f[C]--no-traverse\f[R] to copy only recent 31485 files, eg 31486 .IP 31487 .nf 31488 \f[C] 31489 rclone copy --max-age 24h --no-traverse /path/to/source s3:bucket 31490 \f[R] 31491 .fi 31492 .PP 31493 You\[aq]d then do a full \f[C]rclone sync\f[R] less often. 31494 .PP 31495 Note that \f[C]--fast-list\f[R] isn\[aq]t required in the top-up sync. 31496 .SS Avoiding HEAD requests after PUT 31497 .PP 31498 By default, rclone will HEAD every object it uploads. 31499 It does this to check the object got uploaded correctly. 31500 .PP 31501 You can disable this with the --s3-no-head option - see there for more 31502 details. 31503 .PP 31504 Setting this flag increases the chance for undetected upload failures. 31505 .SS Versions 31506 .PP 31507 When bucket versioning is enabled (this can be done with rclone with the 31508 \f[C]rclone backend versioning\f[R] command) when rclone uploads a new 31509 version of a file it creates a new version of 31510 it (https://docs.aws.amazon.com/AmazonS3/latest/userguide/Versioning.html) 31511 Likewise when you delete a file, the old version will be marked hidden 31512 and still be available. 31513 .PP 31514 Old versions of files, where available, are visible using the 31515 \f[C]--s3-versions\f[R] flag. 31516 .PP 31517 It is also possible to view a bucket as it was at a certain point in 31518 time, using the \f[C]--s3-version-at\f[R] flag. 31519 This will show the file versions as they were at that time, showing 31520 files that have been deleted afterwards, and hiding files that were 31521 created since. 31522 .PP 31523 If you wish to remove all the old versions then you can use the 31524 \f[C]rclone backend cleanup-hidden remote:bucket\f[R] command which will 31525 delete all the old hidden versions of files, leaving the current ones 31526 intact. 31527 You can also supply a path and only old versions under that path will be 31528 deleted, e.g. 31529 \f[C]rclone backend cleanup-hidden remote:bucket/path/to/stuff\f[R]. 31530 .PP 31531 When you \f[C]purge\f[R] a bucket, the current and the old versions will 31532 be deleted then the bucket will be deleted. 31533 .PP 31534 However \f[C]delete\f[R] will cause the current versions of the files to 31535 become hidden old versions. 31536 .PP 31537 Here is a session showing the listing and retrieval of an old version 31538 followed by a \f[C]cleanup\f[R] of the old versions. 31539 .PP 31540 Show current version and all the versions with \f[C]--s3-versions\f[R] 31541 flag. 31542 .IP 31543 .nf 31544 \f[C] 31545 $ rclone -q ls s3:cleanup-test 31546 9 one.txt 31547 31548 $ rclone -q --s3-versions ls s3:cleanup-test 31549 9 one.txt 31550 8 one-v2016-07-04-141032-000.txt 31551 16 one-v2016-07-04-141003-000.txt 31552 15 one-v2016-07-02-155621-000.txt 31553 \f[R] 31554 .fi 31555 .PP 31556 Retrieve an old version 31557 .IP 31558 .nf 31559 \f[C] 31560 $ rclone -q --s3-versions copy s3:cleanup-test/one-v2016-07-04-141003-000.txt /tmp 31561 31562 $ ls -l /tmp/one-v2016-07-04-141003-000.txt 31563 -rw-rw-r-- 1 ncw ncw 16 Jul 2 17:46 /tmp/one-v2016-07-04-141003-000.txt 31564 \f[R] 31565 .fi 31566 .PP 31567 Clean up all the old versions and show that they\[aq]ve gone. 31568 .IP 31569 .nf 31570 \f[C] 31571 $ rclone -q backend cleanup-hidden s3:cleanup-test 31572 31573 $ rclone -q ls s3:cleanup-test 31574 9 one.txt 31575 31576 $ rclone -q --s3-versions ls s3:cleanup-test 31577 9 one.txt 31578 \f[R] 31579 .fi 31580 .SS Versions naming caveat 31581 .PP 31582 When using \f[C]--s3-versions\f[R] flag rclone is relying on the file 31583 name to work out whether the objects are versions or not. 31584 Versions\[aq] names are created by inserting timestamp between file name 31585 and its extension. 31586 .IP 31587 .nf 31588 \f[C] 31589 9 file.txt 31590 8 file-v2023-07-17-161032-000.txt 31591 16 file-v2023-06-15-141003-000.txt 31592 \f[R] 31593 .fi 31594 .PP 31595 If there are real files present with the same names as versions, then 31596 behaviour of \f[C]--s3-versions\f[R] can be unpredictable. 31597 .SS Cleanup 31598 .PP 31599 If you run \f[C]rclone cleanup s3:bucket\f[R] then it will remove all 31600 pending multipart uploads older than 24 hours. 31601 You can use the \f[C]--interactive\f[R]/\f[C]i\f[R] or 31602 \f[C]--dry-run\f[R] flag to see exactly what it will do. 31603 If you want more control over the expiry date then run 31604 \f[C]rclone backend cleanup s3:bucket -o max-age=1h\f[R] to expire all 31605 uploads older than one hour. 31606 You can use \f[C]rclone backend list-multipart-uploads s3:bucket\f[R] to 31607 see the pending multipart uploads. 31608 .SS Restricted filename characters 31609 .PP 31610 S3 allows any valid UTF-8 string as a key. 31611 .PP 31612 Invalid UTF-8 bytes will be 31613 replaced (https://rclone.org/overview/#invalid-utf8), as they can\[aq]t 31614 be used in XML. 31615 .PP 31616 The following characters are replaced since these are problematic when 31617 dealing with the REST API: 31618 .PP 31619 .TS 31620 tab(@); 31621 l c c. 31622 T{ 31623 Character 31624 T}@T{ 31625 Value 31626 T}@T{ 31627 Replacement 31628 T} 31629 _ 31630 T{ 31631 NUL 31632 T}@T{ 31633 0x00 31634 T}@T{ 31635 \[u2400] 31636 T} 31637 T{ 31638 / 31639 T}@T{ 31640 0x2F 31641 T}@T{ 31642 \[uFF0F] 31643 T} 31644 .TE 31645 .PP 31646 The encoding will also encode these file names as they don\[aq]t seem to 31647 work with the SDK properly: 31648 .PP 31649 .TS 31650 tab(@); 31651 l c. 31652 T{ 31653 File name 31654 T}@T{ 31655 Replacement 31656 T} 31657 _ 31658 T{ 31659 \&. 31660 T}@T{ 31661 \[uFF0E] 31662 T} 31663 T{ 31664 \&.. 31665 T}@T{ 31666 \[uFF0E]\[uFF0E] 31667 T} 31668 .TE 31669 .SS Multipart uploads 31670 .PP 31671 rclone supports multipart uploads with S3 which means that it can upload 31672 files bigger than 5 GiB. 31673 .PP 31674 Note that files uploaded \f[I]both\f[R] with multipart upload 31675 \f[I]and\f[R] through crypt remotes do not have MD5 sums. 31676 .PP 31677 rclone switches from single part uploads to multipart uploads at the 31678 point specified by \f[C]--s3-upload-cutoff\f[R]. 31679 This can be a maximum of 5 GiB and a minimum of 0 (ie always upload 31680 multipart files). 31681 .PP 31682 The chunk sizes used in the multipart upload are specified by 31683 \f[C]--s3-chunk-size\f[R] and the number of chunks uploaded concurrently 31684 is specified by \f[C]--s3-upload-concurrency\f[R]. 31685 .PP 31686 Multipart uploads will use \f[C]--transfers\f[R] * 31687 \f[C]--s3-upload-concurrency\f[R] * \f[C]--s3-chunk-size\f[R] extra 31688 memory. 31689 Single part uploads to not use extra memory. 31690 .PP 31691 Single part transfers can be faster than multipart transfers or slower 31692 depending on your latency from S3 - the more latency, the more likely 31693 single part transfers will be faster. 31694 .PP 31695 Increasing \f[C]--s3-upload-concurrency\f[R] will increase throughput (8 31696 would be a sensible value) and increasing \f[C]--s3-chunk-size\f[R] also 31697 increases throughput (16M would be sensible). 31698 Increasing either of these will use more memory. 31699 The default values are high enough to gain most of the possible 31700 performance without using too much memory. 31701 .SS Buckets and Regions 31702 .PP 31703 With Amazon S3 you can list buckets (\f[C]rclone lsd\f[R]) using any 31704 region, but you can only access the content of a bucket from the region 31705 it was created in. 31706 If you attempt to access a bucket from the wrong region, you will get an 31707 error, 31708 \f[C]incorrect region, the bucket is not in \[aq]XXX\[aq] region\f[R]. 31709 .SS Authentication 31710 .PP 31711 There are a number of ways to supply \f[C]rclone\f[R] with a set of AWS 31712 credentials, with and without using the environment. 31713 .PP 31714 The different authentication methods are tried in this order: 31715 .IP \[bu] 2 31716 Directly in the rclone configuration file (\f[C]env_auth = false\f[R] in 31717 the config file): 31718 .RS 2 31719 .IP \[bu] 2 31720 \f[C]access_key_id\f[R] and \f[C]secret_access_key\f[R] are required. 31721 .IP \[bu] 2 31722 \f[C]session_token\f[R] can be optionally set when using AWS STS. 31723 .RE 31724 .IP \[bu] 2 31725 Runtime configuration (\f[C]env_auth = true\f[R] in the config file): 31726 .RS 2 31727 .IP \[bu] 2 31728 Export the following environment variables before running 31729 \f[C]rclone\f[R]: 31730 .RS 2 31731 .IP \[bu] 2 31732 Access Key ID: \f[C]AWS_ACCESS_KEY_ID\f[R] or \f[C]AWS_ACCESS_KEY\f[R] 31733 .IP \[bu] 2 31734 Secret Access Key: \f[C]AWS_SECRET_ACCESS_KEY\f[R] or 31735 \f[C]AWS_SECRET_KEY\f[R] 31736 .IP \[bu] 2 31737 Session Token: \f[C]AWS_SESSION_TOKEN\f[R] (optional) 31738 .RE 31739 .IP \[bu] 2 31740 Or, use a named 31741 profile (https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html): 31742 .RS 2 31743 .IP \[bu] 2 31744 Profile files are standard files used by AWS CLI tools 31745 .IP \[bu] 2 31746 By default it will use the profile in your home directory (e.g. 31747 \f[C]\[ti]/.aws/credentials\f[R] on unix based systems) file and the 31748 \[dq]default\[dq] profile, to change set these environment variables: 31749 .RS 2 31750 .IP \[bu] 2 31751 \f[C]AWS_SHARED_CREDENTIALS_FILE\f[R] to control which file. 31752 .IP \[bu] 2 31753 \f[C]AWS_PROFILE\f[R] to control which profile to use. 31754 .RE 31755 .RE 31756 .IP \[bu] 2 31757 Or, run \f[C]rclone\f[R] in an ECS task with an IAM role (AWS only). 31758 .IP \[bu] 2 31759 Or, run \f[C]rclone\f[R] on an EC2 instance with an IAM role (AWS only). 31760 .IP \[bu] 2 31761 Or, run \f[C]rclone\f[R] in an EKS pod with an IAM role that is 31762 associated with a service account (AWS only). 31763 .RE 31764 .PP 31765 If none of these option actually end up providing \f[C]rclone\f[R] with 31766 AWS credentials then S3 interaction will be non-authenticated (see 31767 below). 31768 .SS S3 Permissions 31769 .PP 31770 When using the \f[C]sync\f[R] subcommand of \f[C]rclone\f[R] the 31771 following minimum permissions are required to be available on the bucket 31772 being written to: 31773 .IP \[bu] 2 31774 \f[C]ListBucket\f[R] 31775 .IP \[bu] 2 31776 \f[C]DeleteObject\f[R] 31777 .IP \[bu] 2 31778 \f[C]GetObject\f[R] 31779 .IP \[bu] 2 31780 \f[C]PutObject\f[R] 31781 .IP \[bu] 2 31782 \f[C]PutObjectACL\f[R] 31783 .IP \[bu] 2 31784 \f[C]CreateBucket\f[R] (unless using s3-no-check-bucket) 31785 .PP 31786 When using the \f[C]lsd\f[R] subcommand, the \f[C]ListAllMyBuckets\f[R] 31787 permission is required. 31788 .PP 31789 Example policy: 31790 .IP 31791 .nf 31792 \f[C] 31793 { 31794 \[dq]Version\[dq]: \[dq]2012-10-17\[dq], 31795 \[dq]Statement\[dq]: [ 31796 { 31797 \[dq]Effect\[dq]: \[dq]Allow\[dq], 31798 \[dq]Principal\[dq]: { 31799 \[dq]AWS\[dq]: \[dq]arn:aws:iam::USER_SID:user/USER_NAME\[dq] 31800 }, 31801 \[dq]Action\[dq]: [ 31802 \[dq]s3:ListBucket\[dq], 31803 \[dq]s3:DeleteObject\[dq], 31804 \[dq]s3:GetObject\[dq], 31805 \[dq]s3:PutObject\[dq], 31806 \[dq]s3:PutObjectAcl\[dq] 31807 ], 31808 \[dq]Resource\[dq]: [ 31809 \[dq]arn:aws:s3:::BUCKET_NAME/*\[dq], 31810 \[dq]arn:aws:s3:::BUCKET_NAME\[dq] 31811 ] 31812 }, 31813 { 31814 \[dq]Effect\[dq]: \[dq]Allow\[dq], 31815 \[dq]Action\[dq]: \[dq]s3:ListAllMyBuckets\[dq], 31816 \[dq]Resource\[dq]: \[dq]arn:aws:s3:::*\[dq] 31817 } 31818 ] 31819 } 31820 \f[R] 31821 .fi 31822 .PP 31823 Notes on above: 31824 .IP "1." 3 31825 This is a policy that can be used when creating bucket. 31826 It assumes that \f[C]USER_NAME\f[R] has been created. 31827 .IP "2." 3 31828 The Resource entry must include both resource ARNs, as one implies the 31829 bucket and the other implies the bucket\[aq]s objects. 31830 .IP "3." 3 31831 When using s3-no-check-bucket and the bucket already exsits, the 31832 \f[C]\[dq]arn:aws:s3:::BUCKET_NAME\[dq]\f[R] doesn\[aq]t have to be 31833 included. 31834 .PP 31835 For reference, here\[aq]s an Ansible 31836 script (https://gist.github.com/ebridges/ebfc9042dd7c756cd101cfa807b7ae2b) 31837 that will generate one or more buckets that will work with 31838 \f[C]rclone sync\f[R]. 31839 .SS Key Management System (KMS) 31840 .PP 31841 If you are using server-side encryption with KMS then you must make sure 31842 rclone is configured with \f[C]server_side_encryption = aws:kms\f[R] 31843 otherwise you will find you can\[aq]t transfer small objects - these 31844 will create checksum errors. 31845 .SS Glacier and Glacier Deep Archive 31846 .PP 31847 You can upload objects using the glacier storage class or transition 31848 them to glacier using a lifecycle 31849 policy (http://docs.aws.amazon.com/AmazonS3/latest/user-guide/create-lifecycle.html). 31850 The bucket can still be synced or copied into normally, but if rclone 31851 tries to access data from the glacier storage class you will see an 31852 error like below. 31853 .IP 31854 .nf 31855 \f[C] 31856 2017/09/11 19:07:43 Failed to sync: failed to open source object: Object in GLACIER, restore first: path/to/file 31857 \f[R] 31858 .fi 31859 .PP 31860 In this case you need to 31861 restore (http://docs.aws.amazon.com/AmazonS3/latest/user-guide/restore-archived-objects.html) 31862 the object(s) in question before using rclone. 31863 .PP 31864 Note that rclone only speaks the S3 API it does not speak the Glacier 31865 Vault API, so rclone cannot directly access Glacier Vaults. 31866 .SS Object-lock enabled S3 bucket 31867 .PP 31868 According to AWS\[aq]s documentation on S3 Object 31869 Lock (https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lock-overview.html#object-lock-permission): 31870 .RS 31871 .PP 31872 If you configure a default retention period on a bucket, requests to 31873 upload objects in such a bucket must include the Content-MD5 header. 31874 .RE 31875 .PP 31876 As mentioned in the Modification times and hashes section, small files 31877 that are not uploaded as multipart, use a different tag, causing the 31878 upload to fail. 31879 A simple solution is to set the \f[C]--s3-upload-cutoff 0\f[R] and force 31880 all the files to be uploaded as multipart. 31881 .SS Standard options 31882 .PP 31883 Here are the Standard options specific to s3 (Amazon S3 Compliant 31884 Storage Providers including AWS, Alibaba, ArvanCloud, Ceph, ChinaMobile, 31885 Cloudflare, DigitalOcean, Dreamhost, GCS, HuaweiOBS, IBMCOS, IDrive, 31886 IONOS, LyveCloud, Leviia, Liara, Linode, Minio, Netease, Petabox, 31887 RackCorp, Rclone, Scaleway, SeaweedFS, StackPath, Storj, Synology, 31888 TencentCOS, Wasabi, Qiniu and others). 31889 .SS --s3-provider 31890 .PP 31891 Choose your S3 provider. 31892 .PP 31893 Properties: 31894 .IP \[bu] 2 31895 Config: provider 31896 .IP \[bu] 2 31897 Env Var: RCLONE_S3_PROVIDER 31898 .IP \[bu] 2 31899 Type: string 31900 .IP \[bu] 2 31901 Required: false 31902 .IP \[bu] 2 31903 Examples: 31904 .RS 2 31905 .IP \[bu] 2 31906 \[dq]AWS\[dq] 31907 .RS 2 31908 .IP \[bu] 2 31909 Amazon Web Services (AWS) S3 31910 .RE 31911 .IP \[bu] 2 31912 \[dq]Alibaba\[dq] 31913 .RS 2 31914 .IP \[bu] 2 31915 Alibaba Cloud Object Storage System (OSS) formerly Aliyun 31916 .RE 31917 .IP \[bu] 2 31918 \[dq]ArvanCloud\[dq] 31919 .RS 2 31920 .IP \[bu] 2 31921 Arvan Cloud Object Storage (AOS) 31922 .RE 31923 .IP \[bu] 2 31924 \[dq]Ceph\[dq] 31925 .RS 2 31926 .IP \[bu] 2 31927 Ceph Object Storage 31928 .RE 31929 .IP \[bu] 2 31930 \[dq]ChinaMobile\[dq] 31931 .RS 2 31932 .IP \[bu] 2 31933 China Mobile Ecloud Elastic Object Storage (EOS) 31934 .RE 31935 .IP \[bu] 2 31936 \[dq]Cloudflare\[dq] 31937 .RS 2 31938 .IP \[bu] 2 31939 Cloudflare R2 Storage 31940 .RE 31941 .IP \[bu] 2 31942 \[dq]DigitalOcean\[dq] 31943 .RS 2 31944 .IP \[bu] 2 31945 DigitalOcean Spaces 31946 .RE 31947 .IP \[bu] 2 31948 \[dq]Dreamhost\[dq] 31949 .RS 2 31950 .IP \[bu] 2 31951 Dreamhost DreamObjects 31952 .RE 31953 .IP \[bu] 2 31954 \[dq]GCS\[dq] 31955 .RS 2 31956 .IP \[bu] 2 31957 Google Cloud Storage 31958 .RE 31959 .IP \[bu] 2 31960 \[dq]HuaweiOBS\[dq] 31961 .RS 2 31962 .IP \[bu] 2 31963 Huawei Object Storage Service 31964 .RE 31965 .IP \[bu] 2 31966 \[dq]IBMCOS\[dq] 31967 .RS 2 31968 .IP \[bu] 2 31969 IBM COS S3 31970 .RE 31971 .IP \[bu] 2 31972 \[dq]IDrive\[dq] 31973 .RS 2 31974 .IP \[bu] 2 31975 IDrive e2 31976 .RE 31977 .IP \[bu] 2 31978 \[dq]IONOS\[dq] 31979 .RS 2 31980 .IP \[bu] 2 31981 IONOS Cloud 31982 .RE 31983 .IP \[bu] 2 31984 \[dq]LyveCloud\[dq] 31985 .RS 2 31986 .IP \[bu] 2 31987 Seagate Lyve Cloud 31988 .RE 31989 .IP \[bu] 2 31990 \[dq]Leviia\[dq] 31991 .RS 2 31992 .IP \[bu] 2 31993 Leviia Object Storage 31994 .RE 31995 .IP \[bu] 2 31996 \[dq]Liara\[dq] 31997 .RS 2 31998 .IP \[bu] 2 31999 Liara Object Storage 32000 .RE 32001 .IP \[bu] 2 32002 \[dq]Linode\[dq] 32003 .RS 2 32004 .IP \[bu] 2 32005 Linode Object Storage 32006 .RE 32007 .IP \[bu] 2 32008 \[dq]Minio\[dq] 32009 .RS 2 32010 .IP \[bu] 2 32011 Minio Object Storage 32012 .RE 32013 .IP \[bu] 2 32014 \[dq]Netease\[dq] 32015 .RS 2 32016 .IP \[bu] 2 32017 Netease Object Storage (NOS) 32018 .RE 32019 .IP \[bu] 2 32020 \[dq]Petabox\[dq] 32021 .RS 2 32022 .IP \[bu] 2 32023 Petabox Object Storage 32024 .RE 32025 .IP \[bu] 2 32026 \[dq]RackCorp\[dq] 32027 .RS 2 32028 .IP \[bu] 2 32029 RackCorp Object Storage 32030 .RE 32031 .IP \[bu] 2 32032 \[dq]Rclone\[dq] 32033 .RS 2 32034 .IP \[bu] 2 32035 Rclone S3 Server 32036 .RE 32037 .IP \[bu] 2 32038 \[dq]Scaleway\[dq] 32039 .RS 2 32040 .IP \[bu] 2 32041 Scaleway Object Storage 32042 .RE 32043 .IP \[bu] 2 32044 \[dq]SeaweedFS\[dq] 32045 .RS 2 32046 .IP \[bu] 2 32047 SeaweedFS S3 32048 .RE 32049 .IP \[bu] 2 32050 \[dq]StackPath\[dq] 32051 .RS 2 32052 .IP \[bu] 2 32053 StackPath Object Storage 32054 .RE 32055 .IP \[bu] 2 32056 \[dq]Storj\[dq] 32057 .RS 2 32058 .IP \[bu] 2 32059 Storj (S3 Compatible Gateway) 32060 .RE 32061 .IP \[bu] 2 32062 \[dq]Synology\[dq] 32063 .RS 2 32064 .IP \[bu] 2 32065 Synology C2 Object Storage 32066 .RE 32067 .IP \[bu] 2 32068 \[dq]TencentCOS\[dq] 32069 .RS 2 32070 .IP \[bu] 2 32071 Tencent Cloud Object Storage (COS) 32072 .RE 32073 .IP \[bu] 2 32074 \[dq]Wasabi\[dq] 32075 .RS 2 32076 .IP \[bu] 2 32077 Wasabi Object Storage 32078 .RE 32079 .IP \[bu] 2 32080 \[dq]Qiniu\[dq] 32081 .RS 2 32082 .IP \[bu] 2 32083 Qiniu Object Storage (Kodo) 32084 .RE 32085 .IP \[bu] 2 32086 \[dq]Other\[dq] 32087 .RS 2 32088 .IP \[bu] 2 32089 Any other S3 compatible provider 32090 .RE 32091 .RE 32092 .SS --s3-env-auth 32093 .PP 32094 Get AWS credentials from runtime (environment variables or EC2/ECS meta 32095 data if no env vars). 32096 .PP 32097 Only applies if access_key_id and secret_access_key is blank. 32098 .PP 32099 Properties: 32100 .IP \[bu] 2 32101 Config: env_auth 32102 .IP \[bu] 2 32103 Env Var: RCLONE_S3_ENV_AUTH 32104 .IP \[bu] 2 32105 Type: bool 32106 .IP \[bu] 2 32107 Default: false 32108 .IP \[bu] 2 32109 Examples: 32110 .RS 2 32111 .IP \[bu] 2 32112 \[dq]false\[dq] 32113 .RS 2 32114 .IP \[bu] 2 32115 Enter AWS credentials in the next step. 32116 .RE 32117 .IP \[bu] 2 32118 \[dq]true\[dq] 32119 .RS 2 32120 .IP \[bu] 2 32121 Get AWS credentials from the environment (env vars or IAM). 32122 .RE 32123 .RE 32124 .SS --s3-access-key-id 32125 .PP 32126 AWS Access Key ID. 32127 .PP 32128 Leave blank for anonymous access or runtime credentials. 32129 .PP 32130 Properties: 32131 .IP \[bu] 2 32132 Config: access_key_id 32133 .IP \[bu] 2 32134 Env Var: RCLONE_S3_ACCESS_KEY_ID 32135 .IP \[bu] 2 32136 Type: string 32137 .IP \[bu] 2 32138 Required: false 32139 .SS --s3-secret-access-key 32140 .PP 32141 AWS Secret Access Key (password). 32142 .PP 32143 Leave blank for anonymous access or runtime credentials. 32144 .PP 32145 Properties: 32146 .IP \[bu] 2 32147 Config: secret_access_key 32148 .IP \[bu] 2 32149 Env Var: RCLONE_S3_SECRET_ACCESS_KEY 32150 .IP \[bu] 2 32151 Type: string 32152 .IP \[bu] 2 32153 Required: false 32154 .SS --s3-region 32155 .PP 32156 Region to connect to. 32157 .PP 32158 Properties: 32159 .IP \[bu] 2 32160 Config: region 32161 .IP \[bu] 2 32162 Env Var: RCLONE_S3_REGION 32163 .IP \[bu] 2 32164 Provider: AWS 32165 .IP \[bu] 2 32166 Type: string 32167 .IP \[bu] 2 32168 Required: false 32169 .IP \[bu] 2 32170 Examples: 32171 .RS 2 32172 .IP \[bu] 2 32173 \[dq]us-east-1\[dq] 32174 .RS 2 32175 .IP \[bu] 2 32176 The default endpoint - a good choice if you are unsure. 32177 .IP \[bu] 2 32178 US Region, Northern Virginia, or Pacific Northwest. 32179 .IP \[bu] 2 32180 Leave location constraint empty. 32181 .RE 32182 .IP \[bu] 2 32183 \[dq]us-east-2\[dq] 32184 .RS 2 32185 .IP \[bu] 2 32186 US East (Ohio) Region. 32187 .IP \[bu] 2 32188 Needs location constraint us-east-2. 32189 .RE 32190 .IP \[bu] 2 32191 \[dq]us-west-1\[dq] 32192 .RS 2 32193 .IP \[bu] 2 32194 US West (Northern California) Region. 32195 .IP \[bu] 2 32196 Needs location constraint us-west-1. 32197 .RE 32198 .IP \[bu] 2 32199 \[dq]us-west-2\[dq] 32200 .RS 2 32201 .IP \[bu] 2 32202 US West (Oregon) Region. 32203 .IP \[bu] 2 32204 Needs location constraint us-west-2. 32205 .RE 32206 .IP \[bu] 2 32207 \[dq]ca-central-1\[dq] 32208 .RS 2 32209 .IP \[bu] 2 32210 Canada (Central) Region. 32211 .IP \[bu] 2 32212 Needs location constraint ca-central-1. 32213 .RE 32214 .IP \[bu] 2 32215 \[dq]eu-west-1\[dq] 32216 .RS 2 32217 .IP \[bu] 2 32218 EU (Ireland) Region. 32219 .IP \[bu] 2 32220 Needs location constraint EU or eu-west-1. 32221 .RE 32222 .IP \[bu] 2 32223 \[dq]eu-west-2\[dq] 32224 .RS 2 32225 .IP \[bu] 2 32226 EU (London) Region. 32227 .IP \[bu] 2 32228 Needs location constraint eu-west-2. 32229 .RE 32230 .IP \[bu] 2 32231 \[dq]eu-west-3\[dq] 32232 .RS 2 32233 .IP \[bu] 2 32234 EU (Paris) Region. 32235 .IP \[bu] 2 32236 Needs location constraint eu-west-3. 32237 .RE 32238 .IP \[bu] 2 32239 \[dq]eu-north-1\[dq] 32240 .RS 2 32241 .IP \[bu] 2 32242 EU (Stockholm) Region. 32243 .IP \[bu] 2 32244 Needs location constraint eu-north-1. 32245 .RE 32246 .IP \[bu] 2 32247 \[dq]eu-south-1\[dq] 32248 .RS 2 32249 .IP \[bu] 2 32250 EU (Milan) Region. 32251 .IP \[bu] 2 32252 Needs location constraint eu-south-1. 32253 .RE 32254 .IP \[bu] 2 32255 \[dq]eu-central-1\[dq] 32256 .RS 2 32257 .IP \[bu] 2 32258 EU (Frankfurt) Region. 32259 .IP \[bu] 2 32260 Needs location constraint eu-central-1. 32261 .RE 32262 .IP \[bu] 2 32263 \[dq]ap-southeast-1\[dq] 32264 .RS 2 32265 .IP \[bu] 2 32266 Asia Pacific (Singapore) Region. 32267 .IP \[bu] 2 32268 Needs location constraint ap-southeast-1. 32269 .RE 32270 .IP \[bu] 2 32271 \[dq]ap-southeast-2\[dq] 32272 .RS 2 32273 .IP \[bu] 2 32274 Asia Pacific (Sydney) Region. 32275 .IP \[bu] 2 32276 Needs location constraint ap-southeast-2. 32277 .RE 32278 .IP \[bu] 2 32279 \[dq]ap-northeast-1\[dq] 32280 .RS 2 32281 .IP \[bu] 2 32282 Asia Pacific (Tokyo) Region. 32283 .IP \[bu] 2 32284 Needs location constraint ap-northeast-1. 32285 .RE 32286 .IP \[bu] 2 32287 \[dq]ap-northeast-2\[dq] 32288 .RS 2 32289 .IP \[bu] 2 32290 Asia Pacific (Seoul). 32291 .IP \[bu] 2 32292 Needs location constraint ap-northeast-2. 32293 .RE 32294 .IP \[bu] 2 32295 \[dq]ap-northeast-3\[dq] 32296 .RS 2 32297 .IP \[bu] 2 32298 Asia Pacific (Osaka-Local). 32299 .IP \[bu] 2 32300 Needs location constraint ap-northeast-3. 32301 .RE 32302 .IP \[bu] 2 32303 \[dq]ap-south-1\[dq] 32304 .RS 2 32305 .IP \[bu] 2 32306 Asia Pacific (Mumbai). 32307 .IP \[bu] 2 32308 Needs location constraint ap-south-1. 32309 .RE 32310 .IP \[bu] 2 32311 \[dq]ap-east-1\[dq] 32312 .RS 2 32313 .IP \[bu] 2 32314 Asia Pacific (Hong Kong) Region. 32315 .IP \[bu] 2 32316 Needs location constraint ap-east-1. 32317 .RE 32318 .IP \[bu] 2 32319 \[dq]sa-east-1\[dq] 32320 .RS 2 32321 .IP \[bu] 2 32322 South America (Sao Paulo) Region. 32323 .IP \[bu] 2 32324 Needs location constraint sa-east-1. 32325 .RE 32326 .IP \[bu] 2 32327 \[dq]me-south-1\[dq] 32328 .RS 2 32329 .IP \[bu] 2 32330 Middle East (Bahrain) Region. 32331 .IP \[bu] 2 32332 Needs location constraint me-south-1. 32333 .RE 32334 .IP \[bu] 2 32335 \[dq]af-south-1\[dq] 32336 .RS 2 32337 .IP \[bu] 2 32338 Africa (Cape Town) Region. 32339 .IP \[bu] 2 32340 Needs location constraint af-south-1. 32341 .RE 32342 .IP \[bu] 2 32343 \[dq]cn-north-1\[dq] 32344 .RS 2 32345 .IP \[bu] 2 32346 China (Beijing) Region. 32347 .IP \[bu] 2 32348 Needs location constraint cn-north-1. 32349 .RE 32350 .IP \[bu] 2 32351 \[dq]cn-northwest-1\[dq] 32352 .RS 2 32353 .IP \[bu] 2 32354 China (Ningxia) Region. 32355 .IP \[bu] 2 32356 Needs location constraint cn-northwest-1. 32357 .RE 32358 .IP \[bu] 2 32359 \[dq]us-gov-east-1\[dq] 32360 .RS 2 32361 .IP \[bu] 2 32362 AWS GovCloud (US-East) Region. 32363 .IP \[bu] 2 32364 Needs location constraint us-gov-east-1. 32365 .RE 32366 .IP \[bu] 2 32367 \[dq]us-gov-west-1\[dq] 32368 .RS 2 32369 .IP \[bu] 2 32370 AWS GovCloud (US) Region. 32371 .IP \[bu] 2 32372 Needs location constraint us-gov-west-1. 32373 .RE 32374 .RE 32375 .SS --s3-endpoint 32376 .PP 32377 Endpoint for S3 API. 32378 .PP 32379 Leave blank if using AWS to use the default endpoint for the region. 32380 .PP 32381 Properties: 32382 .IP \[bu] 2 32383 Config: endpoint 32384 .IP \[bu] 2 32385 Env Var: RCLONE_S3_ENDPOINT 32386 .IP \[bu] 2 32387 Provider: AWS 32388 .IP \[bu] 2 32389 Type: string 32390 .IP \[bu] 2 32391 Required: false 32392 .SS --s3-location-constraint 32393 .PP 32394 Location constraint - must be set to match the Region. 32395 .PP 32396 Used when creating buckets only. 32397 .PP 32398 Properties: 32399 .IP \[bu] 2 32400 Config: location_constraint 32401 .IP \[bu] 2 32402 Env Var: RCLONE_S3_LOCATION_CONSTRAINT 32403 .IP \[bu] 2 32404 Provider: AWS 32405 .IP \[bu] 2 32406 Type: string 32407 .IP \[bu] 2 32408 Required: false 32409 .IP \[bu] 2 32410 Examples: 32411 .RS 2 32412 .IP \[bu] 2 32413 \[dq]\[dq] 32414 .RS 2 32415 .IP \[bu] 2 32416 Empty for US Region, Northern Virginia, or Pacific Northwest 32417 .RE 32418 .IP \[bu] 2 32419 \[dq]us-east-2\[dq] 32420 .RS 2 32421 .IP \[bu] 2 32422 US East (Ohio) Region 32423 .RE 32424 .IP \[bu] 2 32425 \[dq]us-west-1\[dq] 32426 .RS 2 32427 .IP \[bu] 2 32428 US West (Northern California) Region 32429 .RE 32430 .IP \[bu] 2 32431 \[dq]us-west-2\[dq] 32432 .RS 2 32433 .IP \[bu] 2 32434 US West (Oregon) Region 32435 .RE 32436 .IP \[bu] 2 32437 \[dq]ca-central-1\[dq] 32438 .RS 2 32439 .IP \[bu] 2 32440 Canada (Central) Region 32441 .RE 32442 .IP \[bu] 2 32443 \[dq]eu-west-1\[dq] 32444 .RS 2 32445 .IP \[bu] 2 32446 EU (Ireland) Region 32447 .RE 32448 .IP \[bu] 2 32449 \[dq]eu-west-2\[dq] 32450 .RS 2 32451 .IP \[bu] 2 32452 EU (London) Region 32453 .RE 32454 .IP \[bu] 2 32455 \[dq]eu-west-3\[dq] 32456 .RS 2 32457 .IP \[bu] 2 32458 EU (Paris) Region 32459 .RE 32460 .IP \[bu] 2 32461 \[dq]eu-north-1\[dq] 32462 .RS 2 32463 .IP \[bu] 2 32464 EU (Stockholm) Region 32465 .RE 32466 .IP \[bu] 2 32467 \[dq]eu-south-1\[dq] 32468 .RS 2 32469 .IP \[bu] 2 32470 EU (Milan) Region 32471 .RE 32472 .IP \[bu] 2 32473 \[dq]EU\[dq] 32474 .RS 2 32475 .IP \[bu] 2 32476 EU Region 32477 .RE 32478 .IP \[bu] 2 32479 \[dq]ap-southeast-1\[dq] 32480 .RS 2 32481 .IP \[bu] 2 32482 Asia Pacific (Singapore) Region 32483 .RE 32484 .IP \[bu] 2 32485 \[dq]ap-southeast-2\[dq] 32486 .RS 2 32487 .IP \[bu] 2 32488 Asia Pacific (Sydney) Region 32489 .RE 32490 .IP \[bu] 2 32491 \[dq]ap-northeast-1\[dq] 32492 .RS 2 32493 .IP \[bu] 2 32494 Asia Pacific (Tokyo) Region 32495 .RE 32496 .IP \[bu] 2 32497 \[dq]ap-northeast-2\[dq] 32498 .RS 2 32499 .IP \[bu] 2 32500 Asia Pacific (Seoul) Region 32501 .RE 32502 .IP \[bu] 2 32503 \[dq]ap-northeast-3\[dq] 32504 .RS 2 32505 .IP \[bu] 2 32506 Asia Pacific (Osaka-Local) Region 32507 .RE 32508 .IP \[bu] 2 32509 \[dq]ap-south-1\[dq] 32510 .RS 2 32511 .IP \[bu] 2 32512 Asia Pacific (Mumbai) Region 32513 .RE 32514 .IP \[bu] 2 32515 \[dq]ap-east-1\[dq] 32516 .RS 2 32517 .IP \[bu] 2 32518 Asia Pacific (Hong Kong) Region 32519 .RE 32520 .IP \[bu] 2 32521 \[dq]sa-east-1\[dq] 32522 .RS 2 32523 .IP \[bu] 2 32524 South America (Sao Paulo) Region 32525 .RE 32526 .IP \[bu] 2 32527 \[dq]me-south-1\[dq] 32528 .RS 2 32529 .IP \[bu] 2 32530 Middle East (Bahrain) Region 32531 .RE 32532 .IP \[bu] 2 32533 \[dq]af-south-1\[dq] 32534 .RS 2 32535 .IP \[bu] 2 32536 Africa (Cape Town) Region 32537 .RE 32538 .IP \[bu] 2 32539 \[dq]cn-north-1\[dq] 32540 .RS 2 32541 .IP \[bu] 2 32542 China (Beijing) Region 32543 .RE 32544 .IP \[bu] 2 32545 \[dq]cn-northwest-1\[dq] 32546 .RS 2 32547 .IP \[bu] 2 32548 China (Ningxia) Region 32549 .RE 32550 .IP \[bu] 2 32551 \[dq]us-gov-east-1\[dq] 32552 .RS 2 32553 .IP \[bu] 2 32554 AWS GovCloud (US-East) Region 32555 .RE 32556 .IP \[bu] 2 32557 \[dq]us-gov-west-1\[dq] 32558 .RS 2 32559 .IP \[bu] 2 32560 AWS GovCloud (US) Region 32561 .RE 32562 .RE 32563 .SS --s3-acl 32564 .PP 32565 Canned ACL used when creating buckets and storing or copying objects. 32566 .PP 32567 This ACL is used for creating objects and if bucket_acl isn\[aq]t set, 32568 for creating buckets too. 32569 .PP 32570 For more info visit 32571 https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl 32572 .PP 32573 Note that this ACL is applied when server-side copying objects as S3 32574 doesn\[aq]t copy the ACL from the source but rather writes a fresh one. 32575 .PP 32576 If the acl is an empty string then no X-Amz-Acl: header is added and the 32577 default (private) will be used. 32578 .PP 32579 Properties: 32580 .IP \[bu] 2 32581 Config: acl 32582 .IP \[bu] 2 32583 Env Var: RCLONE_S3_ACL 32584 .IP \[bu] 2 32585 Provider: !Storj,Synology,Cloudflare 32586 .IP \[bu] 2 32587 Type: string 32588 .IP \[bu] 2 32589 Required: false 32590 .IP \[bu] 2 32591 Examples: 32592 .RS 2 32593 .IP \[bu] 2 32594 \[dq]default\[dq] 32595 .RS 2 32596 .IP \[bu] 2 32597 Owner gets Full_CONTROL. 32598 .IP \[bu] 2 32599 No one else has access rights (default). 32600 .RE 32601 .IP \[bu] 2 32602 \[dq]private\[dq] 32603 .RS 2 32604 .IP \[bu] 2 32605 Owner gets FULL_CONTROL. 32606 .IP \[bu] 2 32607 No one else has access rights (default). 32608 .RE 32609 .IP \[bu] 2 32610 \[dq]public-read\[dq] 32611 .RS 2 32612 .IP \[bu] 2 32613 Owner gets FULL_CONTROL. 32614 .IP \[bu] 2 32615 The AllUsers group gets READ access. 32616 .RE 32617 .IP \[bu] 2 32618 \[dq]public-read-write\[dq] 32619 .RS 2 32620 .IP \[bu] 2 32621 Owner gets FULL_CONTROL. 32622 .IP \[bu] 2 32623 The AllUsers group gets READ and WRITE access. 32624 .IP \[bu] 2 32625 Granting this on a bucket is generally not recommended. 32626 .RE 32627 .IP \[bu] 2 32628 \[dq]authenticated-read\[dq] 32629 .RS 2 32630 .IP \[bu] 2 32631 Owner gets FULL_CONTROL. 32632 .IP \[bu] 2 32633 The AuthenticatedUsers group gets READ access. 32634 .RE 32635 .IP \[bu] 2 32636 \[dq]bucket-owner-read\[dq] 32637 .RS 2 32638 .IP \[bu] 2 32639 Object owner gets FULL_CONTROL. 32640 .IP \[bu] 2 32641 Bucket owner gets READ access. 32642 .IP \[bu] 2 32643 If you specify this canned ACL when creating a bucket, Amazon S3 ignores 32644 it. 32645 .RE 32646 .IP \[bu] 2 32647 \[dq]bucket-owner-full-control\[dq] 32648 .RS 2 32649 .IP \[bu] 2 32650 Both the object owner and the bucket owner get FULL_CONTROL over the 32651 object. 32652 .IP \[bu] 2 32653 If you specify this canned ACL when creating a bucket, Amazon S3 ignores 32654 it. 32655 .RE 32656 .IP \[bu] 2 32657 \[dq]private\[dq] 32658 .RS 2 32659 .IP \[bu] 2 32660 Owner gets FULL_CONTROL. 32661 .IP \[bu] 2 32662 No one else has access rights (default). 32663 .IP \[bu] 2 32664 This acl is available on IBM Cloud (Infra), IBM Cloud (Storage), 32665 On-Premise COS. 32666 .RE 32667 .IP \[bu] 2 32668 \[dq]public-read\[dq] 32669 .RS 2 32670 .IP \[bu] 2 32671 Owner gets FULL_CONTROL. 32672 .IP \[bu] 2 32673 The AllUsers group gets READ access. 32674 .IP \[bu] 2 32675 This acl is available on IBM Cloud (Infra), IBM Cloud (Storage), 32676 On-Premise IBM COS. 32677 .RE 32678 .IP \[bu] 2 32679 \[dq]public-read-write\[dq] 32680 .RS 2 32681 .IP \[bu] 2 32682 Owner gets FULL_CONTROL. 32683 .IP \[bu] 2 32684 The AllUsers group gets READ and WRITE access. 32685 .IP \[bu] 2 32686 This acl is available on IBM Cloud (Infra), On-Premise IBM COS. 32687 .RE 32688 .IP \[bu] 2 32689 \[dq]authenticated-read\[dq] 32690 .RS 2 32691 .IP \[bu] 2 32692 Owner gets FULL_CONTROL. 32693 .IP \[bu] 2 32694 The AuthenticatedUsers group gets READ access. 32695 .IP \[bu] 2 32696 Not supported on Buckets. 32697 .IP \[bu] 2 32698 This acl is available on IBM Cloud (Infra) and On-Premise IBM COS. 32699 .RE 32700 .RE 32701 .SS --s3-server-side-encryption 32702 .PP 32703 The server-side encryption algorithm used when storing this object in 32704 S3. 32705 .PP 32706 Properties: 32707 .IP \[bu] 2 32708 Config: server_side_encryption 32709 .IP \[bu] 2 32710 Env Var: RCLONE_S3_SERVER_SIDE_ENCRYPTION 32711 .IP \[bu] 2 32712 Provider: AWS,Ceph,ChinaMobile,Minio 32713 .IP \[bu] 2 32714 Type: string 32715 .IP \[bu] 2 32716 Required: false 32717 .IP \[bu] 2 32718 Examples: 32719 .RS 2 32720 .IP \[bu] 2 32721 \[dq]\[dq] 32722 .RS 2 32723 .IP \[bu] 2 32724 None 32725 .RE 32726 .IP \[bu] 2 32727 \[dq]AES256\[dq] 32728 .RS 2 32729 .IP \[bu] 2 32730 AES256 32731 .RE 32732 .IP \[bu] 2 32733 \[dq]aws:kms\[dq] 32734 .RS 2 32735 .IP \[bu] 2 32736 aws:kms 32737 .RE 32738 .RE 32739 .SS --s3-sse-kms-key-id 32740 .PP 32741 If using KMS ID you must provide the ARN of Key. 32742 .PP 32743 Properties: 32744 .IP \[bu] 2 32745 Config: sse_kms_key_id 32746 .IP \[bu] 2 32747 Env Var: RCLONE_S3_SSE_KMS_KEY_ID 32748 .IP \[bu] 2 32749 Provider: AWS,Ceph,Minio 32750 .IP \[bu] 2 32751 Type: string 32752 .IP \[bu] 2 32753 Required: false 32754 .IP \[bu] 2 32755 Examples: 32756 .RS 2 32757 .IP \[bu] 2 32758 \[dq]\[dq] 32759 .RS 2 32760 .IP \[bu] 2 32761 None 32762 .RE 32763 .IP \[bu] 2 32764 \[dq]arn:aws:kms:us-east-1:*\[dq] 32765 .RS 2 32766 .IP \[bu] 2 32767 arn:aws:kms:* 32768 .RE 32769 .RE 32770 .SS --s3-storage-class 32771 .PP 32772 The storage class to use when storing new objects in S3. 32773 .PP 32774 Properties: 32775 .IP \[bu] 2 32776 Config: storage_class 32777 .IP \[bu] 2 32778 Env Var: RCLONE_S3_STORAGE_CLASS 32779 .IP \[bu] 2 32780 Provider: AWS 32781 .IP \[bu] 2 32782 Type: string 32783 .IP \[bu] 2 32784 Required: false 32785 .IP \[bu] 2 32786 Examples: 32787 .RS 2 32788 .IP \[bu] 2 32789 \[dq]\[dq] 32790 .RS 2 32791 .IP \[bu] 2 32792 Default 32793 .RE 32794 .IP \[bu] 2 32795 \[dq]STANDARD\[dq] 32796 .RS 2 32797 .IP \[bu] 2 32798 Standard storage class 32799 .RE 32800 .IP \[bu] 2 32801 \[dq]REDUCED_REDUNDANCY\[dq] 32802 .RS 2 32803 .IP \[bu] 2 32804 Reduced redundancy storage class 32805 .RE 32806 .IP \[bu] 2 32807 \[dq]STANDARD_IA\[dq] 32808 .RS 2 32809 .IP \[bu] 2 32810 Standard Infrequent Access storage class 32811 .RE 32812 .IP \[bu] 2 32813 \[dq]ONEZONE_IA\[dq] 32814 .RS 2 32815 .IP \[bu] 2 32816 One Zone Infrequent Access storage class 32817 .RE 32818 .IP \[bu] 2 32819 \[dq]GLACIER\[dq] 32820 .RS 2 32821 .IP \[bu] 2 32822 Glacier storage class 32823 .RE 32824 .IP \[bu] 2 32825 \[dq]DEEP_ARCHIVE\[dq] 32826 .RS 2 32827 .IP \[bu] 2 32828 Glacier Deep Archive storage class 32829 .RE 32830 .IP \[bu] 2 32831 \[dq]INTELLIGENT_TIERING\[dq] 32832 .RS 2 32833 .IP \[bu] 2 32834 Intelligent-Tiering storage class 32835 .RE 32836 .IP \[bu] 2 32837 \[dq]GLACIER_IR\[dq] 32838 .RS 2 32839 .IP \[bu] 2 32840 Glacier Instant Retrieval storage class 32841 .RE 32842 .RE 32843 .SS Advanced options 32844 .PP 32845 Here are the Advanced options specific to s3 (Amazon S3 Compliant 32846 Storage Providers including AWS, Alibaba, ArvanCloud, Ceph, ChinaMobile, 32847 Cloudflare, DigitalOcean, Dreamhost, GCS, HuaweiOBS, IBMCOS, IDrive, 32848 IONOS, LyveCloud, Leviia, Liara, Linode, Minio, Netease, Petabox, 32849 RackCorp, Rclone, Scaleway, SeaweedFS, StackPath, Storj, Synology, 32850 TencentCOS, Wasabi, Qiniu and others). 32851 .SS --s3-bucket-acl 32852 .PP 32853 Canned ACL used when creating buckets. 32854 .PP 32855 For more info visit 32856 https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl 32857 .PP 32858 Note that this ACL is applied when only when creating buckets. 32859 If it isn\[aq]t set then \[dq]acl\[dq] is used instead. 32860 .PP 32861 If the \[dq]acl\[dq] and \[dq]bucket_acl\[dq] are empty strings then no 32862 X-Amz-Acl: header is added and the default (private) will be used. 32863 .PP 32864 Properties: 32865 .IP \[bu] 2 32866 Config: bucket_acl 32867 .IP \[bu] 2 32868 Env Var: RCLONE_S3_BUCKET_ACL 32869 .IP \[bu] 2 32870 Type: string 32871 .IP \[bu] 2 32872 Required: false 32873 .IP \[bu] 2 32874 Examples: 32875 .RS 2 32876 .IP \[bu] 2 32877 \[dq]private\[dq] 32878 .RS 2 32879 .IP \[bu] 2 32880 Owner gets FULL_CONTROL. 32881 .IP \[bu] 2 32882 No one else has access rights (default). 32883 .RE 32884 .IP \[bu] 2 32885 \[dq]public-read\[dq] 32886 .RS 2 32887 .IP \[bu] 2 32888 Owner gets FULL_CONTROL. 32889 .IP \[bu] 2 32890 The AllUsers group gets READ access. 32891 .RE 32892 .IP \[bu] 2 32893 \[dq]public-read-write\[dq] 32894 .RS 2 32895 .IP \[bu] 2 32896 Owner gets FULL_CONTROL. 32897 .IP \[bu] 2 32898 The AllUsers group gets READ and WRITE access. 32899 .IP \[bu] 2 32900 Granting this on a bucket is generally not recommended. 32901 .RE 32902 .IP \[bu] 2 32903 \[dq]authenticated-read\[dq] 32904 .RS 2 32905 .IP \[bu] 2 32906 Owner gets FULL_CONTROL. 32907 .IP \[bu] 2 32908 The AuthenticatedUsers group gets READ access. 32909 .RE 32910 .RE 32911 .SS --s3-requester-pays 32912 .PP 32913 Enables requester pays option when interacting with S3 bucket. 32914 .PP 32915 Properties: 32916 .IP \[bu] 2 32917 Config: requester_pays 32918 .IP \[bu] 2 32919 Env Var: RCLONE_S3_REQUESTER_PAYS 32920 .IP \[bu] 2 32921 Provider: AWS 32922 .IP \[bu] 2 32923 Type: bool 32924 .IP \[bu] 2 32925 Default: false 32926 .SS --s3-sse-customer-algorithm 32927 .PP 32928 If using SSE-C, the server-side encryption algorithm used when storing 32929 this object in S3. 32930 .PP 32931 Properties: 32932 .IP \[bu] 2 32933 Config: sse_customer_algorithm 32934 .IP \[bu] 2 32935 Env Var: RCLONE_S3_SSE_CUSTOMER_ALGORITHM 32936 .IP \[bu] 2 32937 Provider: AWS,Ceph,ChinaMobile,Minio 32938 .IP \[bu] 2 32939 Type: string 32940 .IP \[bu] 2 32941 Required: false 32942 .IP \[bu] 2 32943 Examples: 32944 .RS 2 32945 .IP \[bu] 2 32946 \[dq]\[dq] 32947 .RS 2 32948 .IP \[bu] 2 32949 None 32950 .RE 32951 .IP \[bu] 2 32952 \[dq]AES256\[dq] 32953 .RS 2 32954 .IP \[bu] 2 32955 AES256 32956 .RE 32957 .RE 32958 .SS --s3-sse-customer-key 32959 .PP 32960 To use SSE-C you may provide the secret encryption key used to 32961 encrypt/decrypt your data. 32962 .PP 32963 Alternatively you can provide --sse-customer-key-base64. 32964 .PP 32965 Properties: 32966 .IP \[bu] 2 32967 Config: sse_customer_key 32968 .IP \[bu] 2 32969 Env Var: RCLONE_S3_SSE_CUSTOMER_KEY 32970 .IP \[bu] 2 32971 Provider: AWS,Ceph,ChinaMobile,Minio 32972 .IP \[bu] 2 32973 Type: string 32974 .IP \[bu] 2 32975 Required: false 32976 .IP \[bu] 2 32977 Examples: 32978 .RS 2 32979 .IP \[bu] 2 32980 \[dq]\[dq] 32981 .RS 2 32982 .IP \[bu] 2 32983 None 32984 .RE 32985 .RE 32986 .SS --s3-sse-customer-key-base64 32987 .PP 32988 If using SSE-C you must provide the secret encryption key encoded in 32989 base64 format to encrypt/decrypt your data. 32990 .PP 32991 Alternatively you can provide --sse-customer-key. 32992 .PP 32993 Properties: 32994 .IP \[bu] 2 32995 Config: sse_customer_key_base64 32996 .IP \[bu] 2 32997 Env Var: RCLONE_S3_SSE_CUSTOMER_KEY_BASE64 32998 .IP \[bu] 2 32999 Provider: AWS,Ceph,ChinaMobile,Minio 33000 .IP \[bu] 2 33001 Type: string 33002 .IP \[bu] 2 33003 Required: false 33004 .IP \[bu] 2 33005 Examples: 33006 .RS 2 33007 .IP \[bu] 2 33008 \[dq]\[dq] 33009 .RS 2 33010 .IP \[bu] 2 33011 None 33012 .RE 33013 .RE 33014 .SS --s3-sse-customer-key-md5 33015 .PP 33016 If using SSE-C you may provide the secret encryption key MD5 checksum 33017 (optional). 33018 .PP 33019 If you leave it blank, this is calculated automatically from the 33020 sse_customer_key provided. 33021 .PP 33022 Properties: 33023 .IP \[bu] 2 33024 Config: sse_customer_key_md5 33025 .IP \[bu] 2 33026 Env Var: RCLONE_S3_SSE_CUSTOMER_KEY_MD5 33027 .IP \[bu] 2 33028 Provider: AWS,Ceph,ChinaMobile,Minio 33029 .IP \[bu] 2 33030 Type: string 33031 .IP \[bu] 2 33032 Required: false 33033 .IP \[bu] 2 33034 Examples: 33035 .RS 2 33036 .IP \[bu] 2 33037 \[dq]\[dq] 33038 .RS 2 33039 .IP \[bu] 2 33040 None 33041 .RE 33042 .RE 33043 .SS --s3-upload-cutoff 33044 .PP 33045 Cutoff for switching to chunked upload. 33046 .PP 33047 Any files larger than this will be uploaded in chunks of chunk_size. 33048 The minimum is 0 and the maximum is 5 GiB. 33049 .PP 33050 Properties: 33051 .IP \[bu] 2 33052 Config: upload_cutoff 33053 .IP \[bu] 2 33054 Env Var: RCLONE_S3_UPLOAD_CUTOFF 33055 .IP \[bu] 2 33056 Type: SizeSuffix 33057 .IP \[bu] 2 33058 Default: 200Mi 33059 .SS --s3-chunk-size 33060 .PP 33061 Chunk size to use for uploading. 33062 .PP 33063 When uploading files larger than upload_cutoff or files with unknown 33064 size (e.g. 33065 from \[dq]rclone rcat\[dq] or uploaded with \[dq]rclone mount\[dq] or 33066 google photos or google docs) they will be uploaded as multipart uploads 33067 using this chunk size. 33068 .PP 33069 Note that \[dq]--s3-upload-concurrency\[dq] chunks of this size are 33070 buffered in memory per transfer. 33071 .PP 33072 If you are transferring large files over high-speed links and you have 33073 enough memory, then increasing this will speed up the transfers. 33074 .PP 33075 Rclone will automatically increase the chunk size when uploading a large 33076 file of known size to stay below the 10,000 chunks limit. 33077 .PP 33078 Files of unknown size are uploaded with the configured chunk_size. 33079 Since the default chunk size is 5 MiB and there can be at most 10,000 33080 chunks, this means that by default the maximum size of a file you can 33081 stream upload is 48 GiB. 33082 If you wish to stream upload larger files then you will need to increase 33083 chunk_size. 33084 .PP 33085 Increasing the chunk size decreases the accuracy of the progress 33086 statistics displayed with \[dq]-P\[dq] flag. 33087 Rclone treats chunk as sent when it\[aq]s buffered by the AWS SDK, when 33088 in fact it may still be uploading. 33089 A bigger chunk size means a bigger AWS SDK buffer and progress reporting 33090 more deviating from the truth. 33091 .PP 33092 Properties: 33093 .IP \[bu] 2 33094 Config: chunk_size 33095 .IP \[bu] 2 33096 Env Var: RCLONE_S3_CHUNK_SIZE 33097 .IP \[bu] 2 33098 Type: SizeSuffix 33099 .IP \[bu] 2 33100 Default: 5Mi 33101 .SS --s3-max-upload-parts 33102 .PP 33103 Maximum number of parts in a multipart upload. 33104 .PP 33105 This option defines the maximum number of multipart chunks to use when 33106 doing a multipart upload. 33107 .PP 33108 This can be useful if a service does not support the AWS S3 33109 specification of 10,000 chunks. 33110 .PP 33111 Rclone will automatically increase the chunk size when uploading a large 33112 file of a known size to stay below this number of chunks limit. 33113 .PP 33114 Properties: 33115 .IP \[bu] 2 33116 Config: max_upload_parts 33117 .IP \[bu] 2 33118 Env Var: RCLONE_S3_MAX_UPLOAD_PARTS 33119 .IP \[bu] 2 33120 Type: int 33121 .IP \[bu] 2 33122 Default: 10000 33123 .SS --s3-copy-cutoff 33124 .PP 33125 Cutoff for switching to multipart copy. 33126 .PP 33127 Any files larger than this that need to be server-side copied will be 33128 copied in chunks of this size. 33129 .PP 33130 The minimum is 0 and the maximum is 5 GiB. 33131 .PP 33132 Properties: 33133 .IP \[bu] 2 33134 Config: copy_cutoff 33135 .IP \[bu] 2 33136 Env Var: RCLONE_S3_COPY_CUTOFF 33137 .IP \[bu] 2 33138 Type: SizeSuffix 33139 .IP \[bu] 2 33140 Default: 4.656Gi 33141 .SS --s3-disable-checksum 33142 .PP 33143 Don\[aq]t store MD5 checksum with object metadata. 33144 .PP 33145 Normally rclone will calculate the MD5 checksum of the input before 33146 uploading it so it can add it to metadata on the object. 33147 This is great for data integrity checking but can cause long delays for 33148 large files to start uploading. 33149 .PP 33150 Properties: 33151 .IP \[bu] 2 33152 Config: disable_checksum 33153 .IP \[bu] 2 33154 Env Var: RCLONE_S3_DISABLE_CHECKSUM 33155 .IP \[bu] 2 33156 Type: bool 33157 .IP \[bu] 2 33158 Default: false 33159 .SS --s3-shared-credentials-file 33160 .PP 33161 Path to the shared credentials file. 33162 .PP 33163 If env_auth = true then rclone can use a shared credentials file. 33164 .PP 33165 If this variable is empty rclone will look for the 33166 \[dq]AWS_SHARED_CREDENTIALS_FILE\[dq] env variable. 33167 If the env value is empty it will default to the current user\[aq]s home 33168 directory. 33169 .IP 33170 .nf 33171 \f[C] 33172 Linux/OSX: \[dq]$HOME/.aws/credentials\[dq] 33173 Windows: \[dq]%USERPROFILE%\[rs].aws\[rs]credentials\[dq] 33174 \f[R] 33175 .fi 33176 .PP 33177 Properties: 33178 .IP \[bu] 2 33179 Config: shared_credentials_file 33180 .IP \[bu] 2 33181 Env Var: RCLONE_S3_SHARED_CREDENTIALS_FILE 33182 .IP \[bu] 2 33183 Type: string 33184 .IP \[bu] 2 33185 Required: false 33186 .SS --s3-profile 33187 .PP 33188 Profile to use in the shared credentials file. 33189 .PP 33190 If env_auth = true then rclone can use a shared credentials file. 33191 This variable controls which profile is used in that file. 33192 .PP 33193 If empty it will default to the environment variable 33194 \[dq]AWS_PROFILE\[dq] or \[dq]default\[dq] if that environment variable 33195 is also not set. 33196 .PP 33197 Properties: 33198 .IP \[bu] 2 33199 Config: profile 33200 .IP \[bu] 2 33201 Env Var: RCLONE_S3_PROFILE 33202 .IP \[bu] 2 33203 Type: string 33204 .IP \[bu] 2 33205 Required: false 33206 .SS --s3-session-token 33207 .PP 33208 An AWS session token. 33209 .PP 33210 Properties: 33211 .IP \[bu] 2 33212 Config: session_token 33213 .IP \[bu] 2 33214 Env Var: RCLONE_S3_SESSION_TOKEN 33215 .IP \[bu] 2 33216 Type: string 33217 .IP \[bu] 2 33218 Required: false 33219 .SS --s3-upload-concurrency 33220 .PP 33221 Concurrency for multipart uploads and copies. 33222 .PP 33223 This is the number of chunks of the same file that are uploaded 33224 concurrently for multipart uploads and copies. 33225 .PP 33226 If you are uploading small numbers of large files over high-speed links 33227 and these uploads do not fully utilize your bandwidth, then increasing 33228 this may help to speed up the transfers. 33229 .PP 33230 Properties: 33231 .IP \[bu] 2 33232 Config: upload_concurrency 33233 .IP \[bu] 2 33234 Env Var: RCLONE_S3_UPLOAD_CONCURRENCY 33235 .IP \[bu] 2 33236 Type: int 33237 .IP \[bu] 2 33238 Default: 4 33239 .SS --s3-force-path-style 33240 .PP 33241 If true use path style access if false use virtual hosted style. 33242 .PP 33243 If this is true (the default) then rclone will use path style access, if 33244 false then rclone will use virtual path style. 33245 See the AWS S3 33246 docs (https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingBucket.html#access-bucket-intro) 33247 for more info. 33248 .PP 33249 Some providers (e.g. 33250 AWS, Aliyun OSS, Netease COS, or Tencent COS) require this set to false 33251 - rclone will do this automatically based on the provider setting. 33252 .PP 33253 Properties: 33254 .IP \[bu] 2 33255 Config: force_path_style 33256 .IP \[bu] 2 33257 Env Var: RCLONE_S3_FORCE_PATH_STYLE 33258 .IP \[bu] 2 33259 Type: bool 33260 .IP \[bu] 2 33261 Default: true 33262 .SS --s3-v2-auth 33263 .PP 33264 If true use v2 authentication. 33265 .PP 33266 If this is false (the default) then rclone will use v4 authentication. 33267 If it is set then rclone will use v2 authentication. 33268 .PP 33269 Use this only if v4 signatures don\[aq]t work, e.g. 33270 pre Jewel/v10 CEPH. 33271 .PP 33272 Properties: 33273 .IP \[bu] 2 33274 Config: v2_auth 33275 .IP \[bu] 2 33276 Env Var: RCLONE_S3_V2_AUTH 33277 .IP \[bu] 2 33278 Type: bool 33279 .IP \[bu] 2 33280 Default: false 33281 .SS --s3-use-dual-stack 33282 .PP 33283 If true use AWS S3 dual-stack endpoint (IPv6 support). 33284 .PP 33285 See AWS Docs on Dualstack 33286 Endpoints (https://docs.aws.amazon.com/AmazonS3/latest/userguide/dual-stack-endpoints.html) 33287 .PP 33288 Properties: 33289 .IP \[bu] 2 33290 Config: use_dual_stack 33291 .IP \[bu] 2 33292 Env Var: RCLONE_S3_USE_DUAL_STACK 33293 .IP \[bu] 2 33294 Type: bool 33295 .IP \[bu] 2 33296 Default: false 33297 .SS --s3-use-accelerate-endpoint 33298 .PP 33299 If true use the AWS S3 accelerated endpoint. 33300 .PP 33301 See: AWS S3 Transfer 33302 acceleration (https://docs.aws.amazon.com/AmazonS3/latest/dev/transfer-acceleration-examples.html) 33303 .PP 33304 Properties: 33305 .IP \[bu] 2 33306 Config: use_accelerate_endpoint 33307 .IP \[bu] 2 33308 Env Var: RCLONE_S3_USE_ACCELERATE_ENDPOINT 33309 .IP \[bu] 2 33310 Provider: AWS 33311 .IP \[bu] 2 33312 Type: bool 33313 .IP \[bu] 2 33314 Default: false 33315 .SS --s3-leave-parts-on-error 33316 .PP 33317 If true avoid calling abort upload on a failure, leaving all 33318 successfully uploaded parts on S3 for manual recovery. 33319 .PP 33320 It should be set to true for resuming uploads across different sessions. 33321 .PP 33322 WARNING: Storing parts of an incomplete multipart upload counts towards 33323 space usage on S3 and will add additional costs if not cleaned up. 33324 .PP 33325 Properties: 33326 .IP \[bu] 2 33327 Config: leave_parts_on_error 33328 .IP \[bu] 2 33329 Env Var: RCLONE_S3_LEAVE_PARTS_ON_ERROR 33330 .IP \[bu] 2 33331 Provider: AWS 33332 .IP \[bu] 2 33333 Type: bool 33334 .IP \[bu] 2 33335 Default: false 33336 .SS --s3-list-chunk 33337 .PP 33338 Size of listing chunk (response list for each ListObject S3 request). 33339 .PP 33340 This option is also known as \[dq]MaxKeys\[dq], \[dq]max-items\[dq], or 33341 \[dq]page-size\[dq] from the AWS S3 specification. 33342 Most services truncate the response list to 1000 objects even if 33343 requested more than that. 33344 In AWS S3 this is a global maximum and cannot be changed, see AWS 33345 S3 (https://docs.aws.amazon.com/cli/latest/reference/s3/ls.html). 33346 In Ceph, this can be increased with the \[dq]rgw list buckets max 33347 chunk\[dq] option. 33348 .PP 33349 Properties: 33350 .IP \[bu] 2 33351 Config: list_chunk 33352 .IP \[bu] 2 33353 Env Var: RCLONE_S3_LIST_CHUNK 33354 .IP \[bu] 2 33355 Type: int 33356 .IP \[bu] 2 33357 Default: 1000 33358 .SS --s3-list-version 33359 .PP 33360 Version of ListObjects to use: 1,2 or 0 for auto. 33361 .PP 33362 When S3 originally launched it only provided the ListObjects call to 33363 enumerate objects in a bucket. 33364 .PP 33365 However in May 2016 the ListObjectsV2 call was introduced. 33366 This is much higher performance and should be used if at all possible. 33367 .PP 33368 If set to the default, 0, rclone will guess according to the provider 33369 set which list objects method to call. 33370 If it guesses wrong, then it may be set manually here. 33371 .PP 33372 Properties: 33373 .IP \[bu] 2 33374 Config: list_version 33375 .IP \[bu] 2 33376 Env Var: RCLONE_S3_LIST_VERSION 33377 .IP \[bu] 2 33378 Type: int 33379 .IP \[bu] 2 33380 Default: 0 33381 .SS --s3-list-url-encode 33382 .PP 33383 Whether to url encode listings: true/false/unset 33384 .PP 33385 Some providers support URL encoding listings and where this is available 33386 this is more reliable when using control characters in file names. 33387 If this is set to unset (the default) then rclone will choose according 33388 to the provider setting what to apply, but you can override rclone\[aq]s 33389 choice here. 33390 .PP 33391 Properties: 33392 .IP \[bu] 2 33393 Config: list_url_encode 33394 .IP \[bu] 2 33395 Env Var: RCLONE_S3_LIST_URL_ENCODE 33396 .IP \[bu] 2 33397 Type: Tristate 33398 .IP \[bu] 2 33399 Default: unset 33400 .SS --s3-no-check-bucket 33401 .PP 33402 If set, don\[aq]t attempt to check the bucket exists or create it. 33403 .PP 33404 This can be useful when trying to minimise the number of transactions 33405 rclone does if you know the bucket exists already. 33406 .PP 33407 It can also be needed if the user you are using does not have bucket 33408 creation permissions. 33409 Before v1.52.0 this would have passed silently due to a bug. 33410 .PP 33411 Properties: 33412 .IP \[bu] 2 33413 Config: no_check_bucket 33414 .IP \[bu] 2 33415 Env Var: RCLONE_S3_NO_CHECK_BUCKET 33416 .IP \[bu] 2 33417 Type: bool 33418 .IP \[bu] 2 33419 Default: false 33420 .SS --s3-no-head 33421 .PP 33422 If set, don\[aq]t HEAD uploaded objects to check integrity. 33423 .PP 33424 This can be useful when trying to minimise the number of transactions 33425 rclone does. 33426 .PP 33427 Setting it means that if rclone receives a 200 OK message after 33428 uploading an object with PUT then it will assume that it got uploaded 33429 properly. 33430 .PP 33431 In particular it will assume: 33432 .IP \[bu] 2 33433 the metadata, including modtime, storage class and content type was as 33434 uploaded 33435 .IP \[bu] 2 33436 the size was as uploaded 33437 .PP 33438 It reads the following items from the response for a single part PUT: 33439 .IP \[bu] 2 33440 the MD5SUM 33441 .IP \[bu] 2 33442 The uploaded date 33443 .PP 33444 For multipart uploads these items aren\[aq]t read. 33445 .PP 33446 If an source object of unknown length is uploaded then rclone 33447 \f[B]will\f[R] do a HEAD request. 33448 .PP 33449 Setting this flag increases the chance for undetected upload failures, 33450 in particular an incorrect size, so it isn\[aq]t recommended for normal 33451 operation. 33452 In practice the chance of an undetected upload failure is very small 33453 even with this flag. 33454 .PP 33455 Properties: 33456 .IP \[bu] 2 33457 Config: no_head 33458 .IP \[bu] 2 33459 Env Var: RCLONE_S3_NO_HEAD 33460 .IP \[bu] 2 33461 Type: bool 33462 .IP \[bu] 2 33463 Default: false 33464 .SS --s3-no-head-object 33465 .PP 33466 If set, do not do HEAD before GET when getting objects. 33467 .PP 33468 Properties: 33469 .IP \[bu] 2 33470 Config: no_head_object 33471 .IP \[bu] 2 33472 Env Var: RCLONE_S3_NO_HEAD_OBJECT 33473 .IP \[bu] 2 33474 Type: bool 33475 .IP \[bu] 2 33476 Default: false 33477 .SS --s3-encoding 33478 .PP 33479 The encoding for the backend. 33480 .PP 33481 See the encoding section in the 33482 overview (https://rclone.org/overview/#encoding) for more info. 33483 .PP 33484 Properties: 33485 .IP \[bu] 2 33486 Config: encoding 33487 .IP \[bu] 2 33488 Env Var: RCLONE_S3_ENCODING 33489 .IP \[bu] 2 33490 Type: Encoding 33491 .IP \[bu] 2 33492 Default: Slash,InvalidUtf8,Dot 33493 .SS --s3-memory-pool-flush-time 33494 .PP 33495 How often internal memory buffer pools will be flushed. 33496 (no longer used) 33497 .PP 33498 Properties: 33499 .IP \[bu] 2 33500 Config: memory_pool_flush_time 33501 .IP \[bu] 2 33502 Env Var: RCLONE_S3_MEMORY_POOL_FLUSH_TIME 33503 .IP \[bu] 2 33504 Type: Duration 33505 .IP \[bu] 2 33506 Default: 1m0s 33507 .SS --s3-memory-pool-use-mmap 33508 .PP 33509 Whether to use mmap buffers in internal memory pool. 33510 (no longer used) 33511 .PP 33512 Properties: 33513 .IP \[bu] 2 33514 Config: memory_pool_use_mmap 33515 .IP \[bu] 2 33516 Env Var: RCLONE_S3_MEMORY_POOL_USE_MMAP 33517 .IP \[bu] 2 33518 Type: bool 33519 .IP \[bu] 2 33520 Default: false 33521 .SS --s3-disable-http2 33522 .PP 33523 Disable usage of http2 for S3 backends. 33524 .PP 33525 There is currently an unsolved issue with the s3 (specifically minio) 33526 backend and HTTP/2. 33527 HTTP/2 is enabled by default for the s3 backend but can be disabled 33528 here. 33529 When the issue is solved this flag will be removed. 33530 .PP 33531 See: https://github.com/artpar/artpar/issues/4673, 33532 https://github.com/artpar/artpar/issues/3631 33533 .PP 33534 Properties: 33535 .IP \[bu] 2 33536 Config: disable_http2 33537 .IP \[bu] 2 33538 Env Var: RCLONE_S3_DISABLE_HTTP2 33539 .IP \[bu] 2 33540 Type: bool 33541 .IP \[bu] 2 33542 Default: false 33543 .SS --s3-download-url 33544 .PP 33545 Custom endpoint for downloads. 33546 This is usually set to a CloudFront CDN URL as AWS S3 offers cheaper 33547 egress for data downloaded through the CloudFront network. 33548 .PP 33549 Properties: 33550 .IP \[bu] 2 33551 Config: download_url 33552 .IP \[bu] 2 33553 Env Var: RCLONE_S3_DOWNLOAD_URL 33554 .IP \[bu] 2 33555 Type: string 33556 .IP \[bu] 2 33557 Required: false 33558 .SS --s3-directory-markers 33559 .PP 33560 Upload an empty object with a trailing slash when a new directory is 33561 created 33562 .PP 33563 Empty folders are unsupported for bucket based remotes, this option 33564 creates an empty object ending with \[dq]/\[dq], to persist the folder. 33565 .PP 33566 Properties: 33567 .IP \[bu] 2 33568 Config: directory_markers 33569 .IP \[bu] 2 33570 Env Var: RCLONE_S3_DIRECTORY_MARKERS 33571 .IP \[bu] 2 33572 Type: bool 33573 .IP \[bu] 2 33574 Default: false 33575 .SS --s3-use-multipart-etag 33576 .PP 33577 Whether to use ETag in multipart uploads for verification 33578 .PP 33579 This should be true, false or left unset to use the default for the 33580 provider. 33581 .PP 33582 Properties: 33583 .IP \[bu] 2 33584 Config: use_multipart_etag 33585 .IP \[bu] 2 33586 Env Var: RCLONE_S3_USE_MULTIPART_ETAG 33587 .IP \[bu] 2 33588 Type: Tristate 33589 .IP \[bu] 2 33590 Default: unset 33591 .SS --s3-use-presigned-request 33592 .PP 33593 Whether to use a presigned request or PutObject for single part uploads 33594 .PP 33595 If this is false rclone will use PutObject from the AWS SDK to upload an 33596 object. 33597 .PP 33598 Versions of rclone < 1.59 use presigned requests to upload a single part 33599 object and setting this flag to true will re-enable that functionality. 33600 This shouldn\[aq]t be necessary except in exceptional circumstances or 33601 for testing. 33602 .PP 33603 Properties: 33604 .IP \[bu] 2 33605 Config: use_presigned_request 33606 .IP \[bu] 2 33607 Env Var: RCLONE_S3_USE_PRESIGNED_REQUEST 33608 .IP \[bu] 2 33609 Type: bool 33610 .IP \[bu] 2 33611 Default: false 33612 .SS --s3-versions 33613 .PP 33614 Include old versions in directory listings. 33615 .PP 33616 Properties: 33617 .IP \[bu] 2 33618 Config: versions 33619 .IP \[bu] 2 33620 Env Var: RCLONE_S3_VERSIONS 33621 .IP \[bu] 2 33622 Type: bool 33623 .IP \[bu] 2 33624 Default: false 33625 .SS --s3-version-at 33626 .PP 33627 Show file versions as they were at the specified time. 33628 .PP 33629 The parameter should be a date, \[dq]2006-01-02\[dq], datetime 33630 \[dq]2006-01-02 15:04:05\[dq] or a duration for that long ago, eg 33631 \[dq]100d\[dq] or \[dq]1h\[dq]. 33632 .PP 33633 Note that when using this no file write operations are permitted, so you 33634 can\[aq]t upload files or delete them. 33635 .PP 33636 See the time option docs (https://rclone.org/docs/#time-option) for 33637 valid formats. 33638 .PP 33639 Properties: 33640 .IP \[bu] 2 33641 Config: version_at 33642 .IP \[bu] 2 33643 Env Var: RCLONE_S3_VERSION_AT 33644 .IP \[bu] 2 33645 Type: Time 33646 .IP \[bu] 2 33647 Default: off 33648 .SS --s3-version-deleted 33649 .PP 33650 Show deleted file markers when using versions. 33651 .PP 33652 This shows deleted file markers in the listing when using versions. 33653 These will appear as 0 size files. 33654 The only operation which can be performed on them is deletion. 33655 .PP 33656 Deleting a delete marker will reveal the previous version. 33657 .PP 33658 Deleted files will always show with a timestamp. 33659 .PP 33660 Properties: 33661 .IP \[bu] 2 33662 Config: version_deleted 33663 .IP \[bu] 2 33664 Env Var: RCLONE_S3_VERSION_DELETED 33665 .IP \[bu] 2 33666 Type: bool 33667 .IP \[bu] 2 33668 Default: false 33669 .SS --s3-decompress 33670 .PP 33671 If set this will decompress gzip encoded objects. 33672 .PP 33673 It is possible to upload objects to S3 with \[dq]Content-Encoding: 33674 gzip\[dq] set. 33675 Normally rclone will download these files as compressed objects. 33676 .PP 33677 If this flag is set then rclone will decompress these files with 33678 \[dq]Content-Encoding: gzip\[dq] as they are received. 33679 This means that rclone can\[aq]t check the size and hash but the file 33680 contents will be decompressed. 33681 .PP 33682 Properties: 33683 .IP \[bu] 2 33684 Config: decompress 33685 .IP \[bu] 2 33686 Env Var: RCLONE_S3_DECOMPRESS 33687 .IP \[bu] 2 33688 Type: bool 33689 .IP \[bu] 2 33690 Default: false 33691 .SS --s3-might-gzip 33692 .PP 33693 Set this if the backend might gzip objects. 33694 .PP 33695 Normally providers will not alter objects when they are downloaded. 33696 If an object was not uploaded with \f[C]Content-Encoding: gzip\f[R] then 33697 it won\[aq]t be set on download. 33698 .PP 33699 However some providers may gzip objects even if they weren\[aq]t 33700 uploaded with \f[C]Content-Encoding: gzip\f[R] (eg Cloudflare). 33701 .PP 33702 A symptom of this would be receiving errors like 33703 .IP 33704 .nf 33705 \f[C] 33706 ERROR corrupted on transfer: sizes differ NNN vs MMM 33707 \f[R] 33708 .fi 33709 .PP 33710 If you set this flag and rclone downloads an object with 33711 Content-Encoding: gzip set and chunked transfer encoding, then rclone 33712 will decompress the object on the fly. 33713 .PP 33714 If this is set to unset (the default) then rclone will choose according 33715 to the provider setting what to apply, but you can override rclone\[aq]s 33716 choice here. 33717 .PP 33718 Properties: 33719 .IP \[bu] 2 33720 Config: might_gzip 33721 .IP \[bu] 2 33722 Env Var: RCLONE_S3_MIGHT_GZIP 33723 .IP \[bu] 2 33724 Type: Tristate 33725 .IP \[bu] 2 33726 Default: unset 33727 .SS --s3-use-accept-encoding-gzip 33728 .PP 33729 Whether to send \f[C]Accept-Encoding: gzip\f[R] header. 33730 .PP 33731 By default, rclone will append \f[C]Accept-Encoding: gzip\f[R] to the 33732 request to download compressed objects whenever possible. 33733 .PP 33734 However some providers such as Google Cloud Storage may alter the HTTP 33735 headers, breaking the signature of the request. 33736 .PP 33737 A symptom of this would be receiving errors like 33738 .IP 33739 .nf 33740 \f[C] 33741 SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. 33742 \f[R] 33743 .fi 33744 .PP 33745 In this case, you might want to try disabling this option. 33746 .PP 33747 Properties: 33748 .IP \[bu] 2 33749 Config: use_accept_encoding_gzip 33750 .IP \[bu] 2 33751 Env Var: RCLONE_S3_USE_ACCEPT_ENCODING_GZIP 33752 .IP \[bu] 2 33753 Type: Tristate 33754 .IP \[bu] 2 33755 Default: unset 33756 .SS --s3-no-system-metadata 33757 .PP 33758 Suppress setting and reading of system metadata 33759 .PP 33760 Properties: 33761 .IP \[bu] 2 33762 Config: no_system_metadata 33763 .IP \[bu] 2 33764 Env Var: RCLONE_S3_NO_SYSTEM_METADATA 33765 .IP \[bu] 2 33766 Type: bool 33767 .IP \[bu] 2 33768 Default: false 33769 .SS --s3-sts-endpoint 33770 .PP 33771 Endpoint for STS. 33772 .PP 33773 Leave blank if using AWS to use the default endpoint for the region. 33774 .PP 33775 Properties: 33776 .IP \[bu] 2 33777 Config: sts_endpoint 33778 .IP \[bu] 2 33779 Env Var: RCLONE_S3_STS_ENDPOINT 33780 .IP \[bu] 2 33781 Provider: AWS 33782 .IP \[bu] 2 33783 Type: string 33784 .IP \[bu] 2 33785 Required: false 33786 .SS --s3-use-already-exists 33787 .PP 33788 Set if rclone should report BucketAlreadyExists errors on bucket 33789 creation. 33790 .PP 33791 At some point during the evolution of the s3 protocol, AWS started 33792 returning an \f[C]AlreadyOwnedByYou\f[R] error when attempting to create 33793 a bucket that the user already owned, rather than a 33794 \f[C]BucketAlreadyExists\f[R] error. 33795 .PP 33796 Unfortunately exactly what has been implemented by s3 clones is a little 33797 inconsistent, some return \f[C]AlreadyOwnedByYou\f[R], some return 33798 \f[C]BucketAlreadyExists\f[R] and some return no error at all. 33799 .PP 33800 This is important to rclone because it ensures the bucket exists by 33801 creating it on quite a lot of operations (unless 33802 \f[C]--s3-no-check-bucket\f[R] is used). 33803 .PP 33804 If rclone knows the provider can return \f[C]AlreadyOwnedByYou\f[R] or 33805 returns no error then it can report \f[C]BucketAlreadyExists\f[R] errors 33806 when the user attempts to create a bucket not owned by them. 33807 Otherwise rclone ignores the \f[C]BucketAlreadyExists\f[R] error which 33808 can lead to confusion. 33809 .PP 33810 This should be automatically set correctly for all providers rclone 33811 knows about - please make a bug report if not. 33812 .PP 33813 Properties: 33814 .IP \[bu] 2 33815 Config: use_already_exists 33816 .IP \[bu] 2 33817 Env Var: RCLONE_S3_USE_ALREADY_EXISTS 33818 .IP \[bu] 2 33819 Type: Tristate 33820 .IP \[bu] 2 33821 Default: unset 33822 .SS --s3-use-multipart-uploads 33823 .PP 33824 Set if rclone should use multipart uploads. 33825 .PP 33826 You can change this if you want to disable the use of multipart uploads. 33827 This shouldn\[aq]t be necessary in normal operation. 33828 .PP 33829 This should be automatically set correctly for all providers rclone 33830 knows about - please make a bug report if not. 33831 .PP 33832 Properties: 33833 .IP \[bu] 2 33834 Config: use_multipart_uploads 33835 .IP \[bu] 2 33836 Env Var: RCLONE_S3_USE_MULTIPART_UPLOADS 33837 .IP \[bu] 2 33838 Type: Tristate 33839 .IP \[bu] 2 33840 Default: unset 33841 .SS --s3-description 33842 .PP 33843 Description of the remote 33844 .PP 33845 Properties: 33846 .IP \[bu] 2 33847 Config: description 33848 .IP \[bu] 2 33849 Env Var: RCLONE_S3_DESCRIPTION 33850 .IP \[bu] 2 33851 Type: string 33852 .IP \[bu] 2 33853 Required: false 33854 .SS Metadata 33855 .PP 33856 User metadata is stored as x-amz-meta- keys. 33857 S3 metadata keys are case insensitive and are always returned in lower 33858 case. 33859 .PP 33860 Here are the possible system metadata items for the s3 backend. 33861 .PP 33862 .TS 33863 tab(@); 33864 lw(11.1n) lw(11.1n) lw(11.1n) lw(16.6n) lw(20.3n). 33865 T{ 33866 Name 33867 T}@T{ 33868 Help 33869 T}@T{ 33870 Type 33871 T}@T{ 33872 Example 33873 T}@T{ 33874 Read Only 33875 T} 33876 _ 33877 T{ 33878 btime 33879 T}@T{ 33880 Time of file birth (creation) read from Last-Modified header 33881 T}@T{ 33882 RFC 3339 33883 T}@T{ 33884 2006-01-02T15:04:05.999999999Z07:00 33885 T}@T{ 33886 \f[B]Y\f[R] 33887 T} 33888 T{ 33889 cache-control 33890 T}@T{ 33891 Cache-Control header 33892 T}@T{ 33893 string 33894 T}@T{ 33895 no-cache 33896 T}@T{ 33897 N 33898 T} 33899 T{ 33900 content-disposition 33901 T}@T{ 33902 Content-Disposition header 33903 T}@T{ 33904 string 33905 T}@T{ 33906 inline 33907 T}@T{ 33908 N 33909 T} 33910 T{ 33911 content-encoding 33912 T}@T{ 33913 Content-Encoding header 33914 T}@T{ 33915 string 33916 T}@T{ 33917 gzip 33918 T}@T{ 33919 N 33920 T} 33921 T{ 33922 content-language 33923 T}@T{ 33924 Content-Language header 33925 T}@T{ 33926 string 33927 T}@T{ 33928 en-US 33929 T}@T{ 33930 N 33931 T} 33932 T{ 33933 content-type 33934 T}@T{ 33935 Content-Type header 33936 T}@T{ 33937 string 33938 T}@T{ 33939 text/plain 33940 T}@T{ 33941 N 33942 T} 33943 T{ 33944 mtime 33945 T}@T{ 33946 Time of last modification, read from rclone metadata 33947 T}@T{ 33948 RFC 3339 33949 T}@T{ 33950 2006-01-02T15:04:05.999999999Z07:00 33951 T}@T{ 33952 N 33953 T} 33954 T{ 33955 tier 33956 T}@T{ 33957 Tier of the object 33958 T}@T{ 33959 string 33960 T}@T{ 33961 GLACIER 33962 T}@T{ 33963 \f[B]Y\f[R] 33964 T} 33965 .TE 33966 .PP 33967 See the metadata (https://rclone.org/docs/#metadata) docs for more info. 33968 .SS Backend commands 33969 .PP 33970 Here are the commands specific to the s3 backend. 33971 .PP 33972 Run them with 33973 .IP 33974 .nf 33975 \f[C] 33976 rclone backend COMMAND remote: 33977 \f[R] 33978 .fi 33979 .PP 33980 The help below will explain what arguments each command takes. 33981 .PP 33982 See the backend (https://rclone.org/commands/rclone_backend/) command 33983 for more info on how to pass options and arguments. 33984 .PP 33985 These can be run on a running backend using the rc command 33986 backend/command (https://rclone.org/rc/#backend-command). 33987 .SS restore 33988 .PP 33989 Restore objects from GLACIER to normal storage 33990 .IP 33991 .nf 33992 \f[C] 33993 rclone backend restore remote: [options] [<arguments>+] 33994 \f[R] 33995 .fi 33996 .PP 33997 This command can be used to restore one or more objects from GLACIER to 33998 normal storage. 33999 .PP 34000 Usage Examples: 34001 .IP 34002 .nf 34003 \f[C] 34004 rclone backend restore s3:bucket/path/to/object -o priority=PRIORITY -o lifetime=DAYS 34005 rclone backend restore s3:bucket/path/to/directory -o priority=PRIORITY -o lifetime=DAYS 34006 rclone backend restore s3:bucket -o priority=PRIORITY -o lifetime=DAYS 34007 \f[R] 34008 .fi 34009 .PP 34010 This flag also obeys the filters. 34011 Test first with --interactive/-i or --dry-run flags 34012 .IP 34013 .nf 34014 \f[C] 34015 rclone --interactive backend restore --include \[dq]*.txt\[dq] s3:bucket/path -o priority=Standard -o lifetime=1 34016 \f[R] 34017 .fi 34018 .PP 34019 All the objects shown will be marked for restore, then 34020 .IP 34021 .nf 34022 \f[C] 34023 rclone backend restore --include \[dq]*.txt\[dq] s3:bucket/path -o priority=Standard -o lifetime=1 34024 \f[R] 34025 .fi 34026 .PP 34027 It returns a list of status dictionaries with Remote and Status keys. 34028 The Status will be OK if it was successful or an error message if not. 34029 .IP 34030 .nf 34031 \f[C] 34032 [ 34033 { 34034 \[dq]Status\[dq]: \[dq]OK\[dq], 34035 \[dq]Remote\[dq]: \[dq]test.txt\[dq] 34036 }, 34037 { 34038 \[dq]Status\[dq]: \[dq]OK\[dq], 34039 \[dq]Remote\[dq]: \[dq]test/file4.txt\[dq] 34040 } 34041 ] 34042 \f[R] 34043 .fi 34044 .PP 34045 Options: 34046 .IP \[bu] 2 34047 \[dq]description\[dq]: The optional description for the job. 34048 .IP \[bu] 2 34049 \[dq]lifetime\[dq]: Lifetime of the active copy in days 34050 .IP \[bu] 2 34051 \[dq]priority\[dq]: Priority of restore: Standard|Expedited|Bulk 34052 .SS restore-status 34053 .PP 34054 Show the restore status for objects being restored from GLACIER to 34055 normal storage 34056 .IP 34057 .nf 34058 \f[C] 34059 rclone backend restore-status remote: [options] [<arguments>+] 34060 \f[R] 34061 .fi 34062 .PP 34063 This command can be used to show the status for objects being restored 34064 from GLACIER to normal storage. 34065 .PP 34066 Usage Examples: 34067 .IP 34068 .nf 34069 \f[C] 34070 rclone backend restore-status s3:bucket/path/to/object 34071 rclone backend restore-status s3:bucket/path/to/directory 34072 rclone backend restore-status -o all s3:bucket/path/to/directory 34073 \f[R] 34074 .fi 34075 .PP 34076 This command does not obey the filters. 34077 .PP 34078 It returns a list of status dictionaries. 34079 .IP 34080 .nf 34081 \f[C] 34082 [ 34083 { 34084 \[dq]Remote\[dq]: \[dq]file.txt\[dq], 34085 \[dq]VersionID\[dq]: null, 34086 \[dq]RestoreStatus\[dq]: { 34087 \[dq]IsRestoreInProgress\[dq]: true, 34088 \[dq]RestoreExpiryDate\[dq]: \[dq]2023-09-06T12:29:19+01:00\[dq] 34089 }, 34090 \[dq]StorageClass\[dq]: \[dq]GLACIER\[dq] 34091 }, 34092 { 34093 \[dq]Remote\[dq]: \[dq]test.pdf\[dq], 34094 \[dq]VersionID\[dq]: null, 34095 \[dq]RestoreStatus\[dq]: { 34096 \[dq]IsRestoreInProgress\[dq]: false, 34097 \[dq]RestoreExpiryDate\[dq]: \[dq]2023-09-06T12:29:19+01:00\[dq] 34098 }, 34099 \[dq]StorageClass\[dq]: \[dq]DEEP_ARCHIVE\[dq] 34100 } 34101 ] 34102 \f[R] 34103 .fi 34104 .PP 34105 Options: 34106 .IP \[bu] 2 34107 \[dq]all\[dq]: if set then show all objects, not just ones with restore 34108 status 34109 .SS list-multipart-uploads 34110 .PP 34111 List the unfinished multipart uploads 34112 .IP 34113 .nf 34114 \f[C] 34115 rclone backend list-multipart-uploads remote: [options] [<arguments>+] 34116 \f[R] 34117 .fi 34118 .PP 34119 This command lists the unfinished multipart uploads in JSON format. 34120 .IP 34121 .nf 34122 \f[C] 34123 rclone backend list-multipart s3:bucket/path/to/object 34124 \f[R] 34125 .fi 34126 .PP 34127 It returns a dictionary of buckets with values as lists of unfinished 34128 multipart uploads. 34129 .PP 34130 You can call it with no bucket in which case it lists all bucket, with a 34131 bucket or with a bucket and path. 34132 .IP 34133 .nf 34134 \f[C] 34135 { 34136 \[dq]rclone\[dq]: [ 34137 { 34138 \[dq]Initiated\[dq]: \[dq]2020-06-26T14:20:36Z\[dq], 34139 \[dq]Initiator\[dq]: { 34140 \[dq]DisplayName\[dq]: \[dq]XXX\[dq], 34141 \[dq]ID\[dq]: \[dq]arn:aws:iam::XXX:user/XXX\[dq] 34142 }, 34143 \[dq]Key\[dq]: \[dq]KEY\[dq], 34144 \[dq]Owner\[dq]: { 34145 \[dq]DisplayName\[dq]: null, 34146 \[dq]ID\[dq]: \[dq]XXX\[dq] 34147 }, 34148 \[dq]StorageClass\[dq]: \[dq]STANDARD\[dq], 34149 \[dq]UploadId\[dq]: \[dq]XXX\[dq] 34150 } 34151 ], 34152 \[dq]rclone-1000files\[dq]: [], 34153 \[dq]rclone-dst\[dq]: [] 34154 } 34155 \f[R] 34156 .fi 34157 .SS cleanup 34158 .PP 34159 Remove unfinished multipart uploads. 34160 .IP 34161 .nf 34162 \f[C] 34163 rclone backend cleanup remote: [options] [<arguments>+] 34164 \f[R] 34165 .fi 34166 .PP 34167 This command removes unfinished multipart uploads of age greater than 34168 max-age which defaults to 24 hours. 34169 .PP 34170 Note that you can use --interactive/-i or --dry-run with this command to 34171 see what it would do. 34172 .IP 34173 .nf 34174 \f[C] 34175 rclone backend cleanup s3:bucket/path/to/object 34176 rclone backend cleanup -o max-age=7w s3:bucket/path/to/object 34177 \f[R] 34178 .fi 34179 .PP 34180 Durations are parsed as per the rest of rclone, 2h, 7d, 7w etc. 34181 .PP 34182 Options: 34183 .IP \[bu] 2 34184 \[dq]max-age\[dq]: Max age of upload to delete 34185 .SS cleanup-hidden 34186 .PP 34187 Remove old versions of files. 34188 .IP 34189 .nf 34190 \f[C] 34191 rclone backend cleanup-hidden remote: [options] [<arguments>+] 34192 \f[R] 34193 .fi 34194 .PP 34195 This command removes any old hidden versions of files on a versions 34196 enabled bucket. 34197 .PP 34198 Note that you can use --interactive/-i or --dry-run with this command to 34199 see what it would do. 34200 .IP 34201 .nf 34202 \f[C] 34203 rclone backend cleanup-hidden s3:bucket/path/to/dir 34204 \f[R] 34205 .fi 34206 .SS versioning 34207 .PP 34208 Set/get versioning support for a bucket. 34209 .IP 34210 .nf 34211 \f[C] 34212 rclone backend versioning remote: [options] [<arguments>+] 34213 \f[R] 34214 .fi 34215 .PP 34216 This command sets versioning support if a parameter is passed and then 34217 returns the current versioning status for the bucket supplied. 34218 .IP 34219 .nf 34220 \f[C] 34221 rclone backend versioning s3:bucket # read status only 34222 rclone backend versioning s3:bucket Enabled 34223 rclone backend versioning s3:bucket Suspended 34224 \f[R] 34225 .fi 34226 .PP 34227 It may return \[dq]Enabled\[dq], \[dq]Suspended\[dq] or 34228 \[dq]Unversioned\[dq]. 34229 Note that once versioning has been enabled the status can\[aq]t be set 34230 back to \[dq]Unversioned\[dq]. 34231 .SS set 34232 .PP 34233 Set command for updating the config parameters. 34234 .IP 34235 .nf 34236 \f[C] 34237 rclone backend set remote: [options] [<arguments>+] 34238 \f[R] 34239 .fi 34240 .PP 34241 This set command can be used to update the config parameters for a 34242 running s3 backend. 34243 .PP 34244 Usage Examples: 34245 .IP 34246 .nf 34247 \f[C] 34248 rclone backend set s3: [-o opt_name=opt_value] [-o opt_name2=opt_value2] 34249 rclone rc backend/command command=set fs=s3: [-o opt_name=opt_value] [-o opt_name2=opt_value2] 34250 rclone rc backend/command command=set fs=s3: -o session_token=X -o access_key_id=X -o secret_access_key=X 34251 \f[R] 34252 .fi 34253 .PP 34254 The option keys are named as they are in the config file. 34255 .PP 34256 This rebuilds the connection to the s3 backend when it is called with 34257 the new parameters. 34258 Only new parameters need be passed as the values will default to those 34259 currently in use. 34260 .PP 34261 It doesn\[aq]t return anything. 34262 .SS Anonymous access to public buckets 34263 .PP 34264 If you want to use rclone to access a public bucket, configure with a 34265 blank \f[C]access_key_id\f[R] and \f[C]secret_access_key\f[R]. 34266 Your config should end up looking like this: 34267 .IP 34268 .nf 34269 \f[C] 34270 [anons3] 34271 type = s3 34272 provider = AWS 34273 env_auth = false 34274 access_key_id = 34275 secret_access_key = 34276 region = us-east-1 34277 endpoint = 34278 location_constraint = 34279 acl = private 34280 server_side_encryption = 34281 storage_class = 34282 \f[R] 34283 .fi 34284 .PP 34285 Then use it as normal with the name of the public bucket, e.g. 34286 .IP 34287 .nf 34288 \f[C] 34289 rclone lsd anons3:1000genomes 34290 \f[R] 34291 .fi 34292 .PP 34293 You will be able to list and copy data but not upload it. 34294 .SS Providers 34295 .SS AWS S3 34296 .PP 34297 This is the provider used as main example and described in the 34298 configuration section above. 34299 .SS AWS Snowball Edge 34300 .PP 34301 AWS Snowball (https://aws.amazon.com/snowball/) is a hardware appliance 34302 used for transferring bulk data back to AWS. 34303 Its main software interface is S3 object storage. 34304 .PP 34305 To use rclone with AWS Snowball Edge devices, configure as standard for 34306 an \[aq]S3 Compatible Service\[aq]. 34307 .PP 34308 If using rclone pre v1.59 be sure to set \f[C]upload_cutoff = 0\f[R] 34309 otherwise you will run into authentication header issues as the snowball 34310 device does not support query parameter based authentication. 34311 .PP 34312 With rclone v1.59 or later setting \f[C]upload_cutoff\f[R] should not be 34313 necessary. 34314 .PP 34315 eg. 34316 .IP 34317 .nf 34318 \f[C] 34319 [snowball] 34320 type = s3 34321 provider = Other 34322 access_key_id = YOUR_ACCESS_KEY 34323 secret_access_key = YOUR_SECRET_KEY 34324 endpoint = http://[IP of Snowball]:8080 34325 upload_cutoff = 0 34326 \f[R] 34327 .fi 34328 .SS Ceph 34329 .PP 34330 Ceph (https://ceph.com/) is an open-source, unified, distributed storage 34331 system designed for excellent performance, reliability and scalability. 34332 It has an S3 compatible object storage interface. 34333 .PP 34334 To use rclone with Ceph, configure as above but leave the region blank 34335 and set the endpoint. 34336 You should end up with something like this in your config: 34337 .IP 34338 .nf 34339 \f[C] 34340 [ceph] 34341 type = s3 34342 provider = Ceph 34343 env_auth = false 34344 access_key_id = XXX 34345 secret_access_key = YYY 34346 region = 34347 endpoint = https://ceph.endpoint.example.com 34348 location_constraint = 34349 acl = 34350 server_side_encryption = 34351 storage_class = 34352 \f[R] 34353 .fi 34354 .PP 34355 If you are using an older version of CEPH (e.g. 34356 10.2.x Jewel) and a version of rclone before v1.59 then you may need to 34357 supply the parameter \f[C]--s3-upload-cutoff 0\f[R] or put this in the 34358 config file as \f[C]upload_cutoff 0\f[R] to work around a bug which 34359 causes uploading of small files to fail. 34360 .PP 34361 Note also that Ceph sometimes puts \f[C]/\f[R] in the passwords it gives 34362 users. 34363 If you read the secret access key using the command line tools you will 34364 get a JSON blob with the \f[C]/\f[R] escaped as \f[C]\[rs]/\f[R]. 34365 Make sure you only write \f[C]/\f[R] in the secret access key. 34366 .PP 34367 Eg the dump from Ceph looks something like this (irrelevant keys 34368 removed). 34369 .IP 34370 .nf 34371 \f[C] 34372 { 34373 \[dq]user_id\[dq]: \[dq]xxx\[dq], 34374 \[dq]display_name\[dq]: \[dq]xxxx\[dq], 34375 \[dq]keys\[dq]: [ 34376 { 34377 \[dq]user\[dq]: \[dq]xxx\[dq], 34378 \[dq]access_key\[dq]: \[dq]xxxxxx\[dq], 34379 \[dq]secret_key\[dq]: \[dq]xxxxxx\[rs]/xxxx\[dq] 34380 } 34381 ], 34382 } 34383 \f[R] 34384 .fi 34385 .PP 34386 Because this is a json dump, it is encoding the \f[C]/\f[R] as 34387 \f[C]\[rs]/\f[R], so if you use the secret key as \f[C]xxxxxx/xxxx\f[R] 34388 it will work fine. 34389 .SS Cloudflare R2 34390 .PP 34391 Cloudflare R2 (https://blog.cloudflare.com/r2-open-beta/) Storage allows 34392 developers to store large amounts of unstructured data without the 34393 costly egress bandwidth fees associated with typical cloud storage 34394 services. 34395 .PP 34396 Here is an example of making a Cloudflare R2 configuration. 34397 First run: 34398 .IP 34399 .nf 34400 \f[C] 34401 rclone config 34402 \f[R] 34403 .fi 34404 .PP 34405 This will guide you through an interactive setup process. 34406 .PP 34407 Note that all buckets are private, and all are stored in the same 34408 \[dq]auto\[dq] region. 34409 It is necessary to use Cloudflare workers to share the content of a 34410 bucket publicly. 34411 .IP 34412 .nf 34413 \f[C] 34414 No remotes found, make a new one? 34415 n) New remote 34416 s) Set configuration password 34417 q) Quit config 34418 n/s/q> n 34419 name> r2 34420 Option Storage. 34421 Type of storage to configure. 34422 Choose a number from below, or type in your own value. 34423 \&... 34424 XX / Amazon S3 Compliant Storage Providers including AWS, Alibaba, Ceph, China Mobile, Cloudflare, ArvanCloud, DigitalOcean, Dreamhost, Huawei OBS, IBM COS, Lyve Cloud, Minio, Netease, RackCorp, Scaleway, SeaweedFS, StackPath, Storj, Synology, Tencent COS and Wasabi 34425 \[rs] (s3) 34426 \&... 34427 Storage> s3 34428 Option provider. 34429 Choose your S3 provider. 34430 Choose a number from below, or type in your own value. 34431 Press Enter to leave empty. 34432 \&... 34433 XX / Cloudflare R2 Storage 34434 \[rs] (Cloudflare) 34435 \&... 34436 provider> Cloudflare 34437 Option env_auth. 34438 Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars). 34439 Only applies if access_key_id and secret_access_key is blank. 34440 Choose a number from below, or type in your own boolean value (true or false). 34441 Press Enter for the default (false). 34442 1 / Enter AWS credentials in the next step. 34443 \[rs] (false) 34444 2 / Get AWS credentials from the environment (env vars or IAM). 34445 \[rs] (true) 34446 env_auth> 1 34447 Option access_key_id. 34448 AWS Access Key ID. 34449 Leave blank for anonymous access or runtime credentials. 34450 Enter a value. Press Enter to leave empty. 34451 access_key_id> ACCESS_KEY 34452 Option secret_access_key. 34453 AWS Secret Access Key (password). 34454 Leave blank for anonymous access or runtime credentials. 34455 Enter a value. Press Enter to leave empty. 34456 secret_access_key> SECRET_ACCESS_KEY 34457 Option region. 34458 Region to connect to. 34459 Choose a number from below, or type in your own value. 34460 Press Enter to leave empty. 34461 1 / R2 buckets are automatically distributed across Cloudflare\[aq]s data centers for low latency. 34462 \[rs] (auto) 34463 region> 1 34464 Option endpoint. 34465 Endpoint for S3 API. 34466 Required when using an S3 clone. 34467 Enter a value. Press Enter to leave empty. 34468 endpoint> https://ACCOUNT_ID.r2.cloudflarestorage.com 34469 Edit advanced config? 34470 y) Yes 34471 n) No (default) 34472 y/n> n 34473 -------------------- 34474 y) Yes this is OK (default) 34475 e) Edit this remote 34476 d) Delete this remote 34477 y/e/d> y 34478 \f[R] 34479 .fi 34480 .PP 34481 This will leave your config looking something like: 34482 .IP 34483 .nf 34484 \f[C] 34485 [r2] 34486 type = s3 34487 provider = Cloudflare 34488 access_key_id = ACCESS_KEY 34489 secret_access_key = SECRET_ACCESS_KEY 34490 region = auto 34491 endpoint = https://ACCOUNT_ID.r2.cloudflarestorage.com 34492 acl = private 34493 \f[R] 34494 .fi 34495 .PP 34496 Now run \f[C]rclone lsf r2:\f[R] to see your buckets and 34497 \f[C]rclone lsf r2:bucket\f[R] to look within a bucket. 34498 .SS Dreamhost 34499 .PP 34500 Dreamhost DreamObjects (https://www.dreamhost.com/cloud/storage/) is an 34501 object storage system based on CEPH. 34502 .PP 34503 To use rclone with Dreamhost, configure as above but leave the region 34504 blank and set the endpoint. 34505 You should end up with something like this in your config: 34506 .IP 34507 .nf 34508 \f[C] 34509 [dreamobjects] 34510 type = s3 34511 provider = DreamHost 34512 env_auth = false 34513 access_key_id = your_access_key 34514 secret_access_key = your_secret_key 34515 region = 34516 endpoint = objects-us-west-1.dream.io 34517 location_constraint = 34518 acl = private 34519 server_side_encryption = 34520 storage_class = 34521 \f[R] 34522 .fi 34523 .SS Google Cloud Storage 34524 .PP 34525 GoogleCloudStorage (https://cloud.google.com/storage/docs) is an 34526 S3-interoperable (https://cloud.google.com/storage/docs/interoperability) 34527 object storage service from Google Cloud Platform. 34528 .PP 34529 To connect to Google Cloud Storage you will need an access key and 34530 secret key. 34531 These can be retrieved by creating an HMAC 34532 key (https://cloud.google.com/storage/docs/authentication/managing-hmackeys). 34533 .IP 34534 .nf 34535 \f[C] 34536 [gs] 34537 type = s3 34538 provider = GCS 34539 access_key_id = your_access_key 34540 secret_access_key = your_secret_key 34541 endpoint = https://storage.googleapis.com 34542 \f[R] 34543 .fi 34544 .PP 34545 \f[B]Note\f[R] that \f[C]--s3-versions\f[R] does not work with GCS when 34546 it needs to do directory paging. 34547 Rclone will return the error: 34548 .IP 34549 .nf 34550 \f[C] 34551 s3 protocol error: received versions listing with IsTruncated set with no NextKeyMarker 34552 \f[R] 34553 .fi 34554 .PP 34555 This is Google bug 34556 #312292516 (https://issuetracker.google.com/u/0/issues/312292516). 34557 .SS DigitalOcean Spaces 34558 .PP 34559 Spaces (https://www.digitalocean.com/products/object-storage/) is an 34560 S3-interoperable (https://developers.digitalocean.com/documentation/spaces/) 34561 object storage service from cloud provider DigitalOcean. 34562 .PP 34563 To connect to DigitalOcean Spaces you will need an access key and secret 34564 key. 34565 These can be retrieved on the \[dq]Applications & 34566 API (https://cloud.digitalocean.com/settings/api/tokens)\[dq] page of 34567 the DigitalOcean control panel. 34568 They will be needed when prompted by \f[C]rclone config\f[R] for your 34569 \f[C]access_key_id\f[R] and \f[C]secret_access_key\f[R]. 34570 .PP 34571 When prompted for a \f[C]region\f[R] or \f[C]location_constraint\f[R], 34572 press enter to use the default value. 34573 The region must be included in the \f[C]endpoint\f[R] setting (e.g. 34574 \f[C]nyc3.digitaloceanspaces.com\f[R]). 34575 The default values can be used for other settings. 34576 .PP 34577 Going through the whole process of creating a new remote by running 34578 \f[C]rclone config\f[R], each prompt should be answered as shown below: 34579 .IP 34580 .nf 34581 \f[C] 34582 Storage> s3 34583 env_auth> 1 34584 access_key_id> YOUR_ACCESS_KEY 34585 secret_access_key> YOUR_SECRET_KEY 34586 region> 34587 endpoint> nyc3.digitaloceanspaces.com 34588 location_constraint> 34589 acl> 34590 storage_class> 34591 \f[R] 34592 .fi 34593 .PP 34594 The resulting configuration file should look like: 34595 .IP 34596 .nf 34597 \f[C] 34598 [spaces] 34599 type = s3 34600 provider = DigitalOcean 34601 env_auth = false 34602 access_key_id = YOUR_ACCESS_KEY 34603 secret_access_key = YOUR_SECRET_KEY 34604 region = 34605 endpoint = nyc3.digitaloceanspaces.com 34606 location_constraint = 34607 acl = 34608 server_side_encryption = 34609 storage_class = 34610 \f[R] 34611 .fi 34612 .PP 34613 Once configured, you can create a new Space and begin copying files. 34614 For example: 34615 .IP 34616 .nf 34617 \f[C] 34618 rclone mkdir spaces:my-new-space 34619 rclone copy /path/to/files spaces:my-new-space 34620 \f[R] 34621 .fi 34622 .SS Huawei OBS 34623 .PP 34624 Object Storage Service (OBS) provides stable, secure, efficient, and 34625 easy-to-use cloud storage that lets you store virtually any volume of 34626 unstructured data in any format and access it from anywhere. 34627 .PP 34628 OBS provides an S3 interface, you can copy and modify the following 34629 configuration and add it to your rclone configuration file. 34630 .IP 34631 .nf 34632 \f[C] 34633 [obs] 34634 type = s3 34635 provider = HuaweiOBS 34636 access_key_id = your-access-key-id 34637 secret_access_key = your-secret-access-key 34638 region = af-south-1 34639 endpoint = obs.af-south-1.myhuaweicloud.com 34640 acl = private 34641 \f[R] 34642 .fi 34643 .PP 34644 Or you can also configure via the interactive command line: 34645 .IP 34646 .nf 34647 \f[C] 34648 No remotes found, make a new one? 34649 n) New remote 34650 s) Set configuration password 34651 q) Quit config 34652 n/s/q> n 34653 name> obs 34654 Option Storage. 34655 Type of storage to configure. 34656 Choose a number from below, or type in your own value. 34657 [snip] 34658 XX / Amazon S3 Compliant Storage Providers including AWS, ... 34659 \[rs] (s3) 34660 [snip] 34661 Storage> s3 34662 Option provider. 34663 Choose your S3 provider. 34664 Choose a number from below, or type in your own value. 34665 Press Enter to leave empty. 34666 [snip] 34667 9 / Huawei Object Storage Service 34668 \[rs] (HuaweiOBS) 34669 [snip] 34670 provider> 9 34671 Option env_auth. 34672 Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars). 34673 Only applies if access_key_id and secret_access_key is blank. 34674 Choose a number from below, or type in your own boolean value (true or false). 34675 Press Enter for the default (false). 34676 1 / Enter AWS credentials in the next step. 34677 \[rs] (false) 34678 2 / Get AWS credentials from the environment (env vars or IAM). 34679 \[rs] (true) 34680 env_auth> 1 34681 Option access_key_id. 34682 AWS Access Key ID. 34683 Leave blank for anonymous access or runtime credentials. 34684 Enter a value. Press Enter to leave empty. 34685 access_key_id> your-access-key-id 34686 Option secret_access_key. 34687 AWS Secret Access Key (password). 34688 Leave blank for anonymous access or runtime credentials. 34689 Enter a value. Press Enter to leave empty. 34690 secret_access_key> your-secret-access-key 34691 Option region. 34692 Region to connect to. 34693 Choose a number from below, or type in your own value. 34694 Press Enter to leave empty. 34695 1 / AF-Johannesburg 34696 \[rs] (af-south-1) 34697 2 / AP-Bangkok 34698 \[rs] (ap-southeast-2) 34699 [snip] 34700 region> 1 34701 Option endpoint. 34702 Endpoint for OBS API. 34703 Choose a number from below, or type in your own value. 34704 Press Enter to leave empty. 34705 1 / AF-Johannesburg 34706 \[rs] (obs.af-south-1.myhuaweicloud.com) 34707 2 / AP-Bangkok 34708 \[rs] (obs.ap-southeast-2.myhuaweicloud.com) 34709 [snip] 34710 endpoint> 1 34711 Option acl. 34712 Canned ACL used when creating buckets and storing or copying objects. 34713 This ACL is used for creating objects and if bucket_acl isn\[aq]t set, for creating buckets too. 34714 For more info visit https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl 34715 Note that this ACL is applied when server-side copying objects as S3 34716 doesn\[aq]t copy the ACL from the source but rather writes a fresh one. 34717 Choose a number from below, or type in your own value. 34718 Press Enter to leave empty. 34719 / Owner gets FULL_CONTROL. 34720 1 | No one else has access rights (default). 34721 \[rs] (private) 34722 [snip] 34723 acl> 1 34724 Edit advanced config? 34725 y) Yes 34726 n) No (default) 34727 y/n> 34728 -------------------- 34729 [obs] 34730 type = s3 34731 provider = HuaweiOBS 34732 access_key_id = your-access-key-id 34733 secret_access_key = your-secret-access-key 34734 region = af-south-1 34735 endpoint = obs.af-south-1.myhuaweicloud.com 34736 acl = private 34737 -------------------- 34738 y) Yes this is OK (default) 34739 e) Edit this remote 34740 d) Delete this remote 34741 y/e/d> y 34742 Current remotes: 34743 34744 Name Type 34745 ==== ==== 34746 obs s3 34747 34748 e) Edit existing remote 34749 n) New remote 34750 d) Delete remote 34751 r) Rename remote 34752 c) Copy remote 34753 s) Set configuration password 34754 q) Quit config 34755 e/n/d/r/c/s/q> q 34756 \f[R] 34757 .fi 34758 .SS IBM COS (S3) 34759 .PP 34760 Information stored with IBM Cloud Object Storage is encrypted and 34761 dispersed across multiple geographic locations, and accessed through an 34762 implementation of the S3 API. 34763 This service makes use of the distributed storage technologies provided 34764 by IBM\[cq]s Cloud Object Storage System (formerly Cleversafe). 34765 For more information visit: (http://www.ibm.com/cloud/object-storage) 34766 .PP 34767 To configure access to IBM COS S3, follow the steps below: 34768 .IP "1." 3 34769 Run rclone config and select n for a new remote. 34770 .IP 34771 .nf 34772 \f[C] 34773 2018/02/14 14:13:11 NOTICE: Config file \[dq]C:\[rs]\[rs]Users\[rs]\[rs]a\[rs]\[rs].config\[rs]\[rs]rclone\[rs]\[rs]rclone.conf\[dq] not found - using defaults 34774 No remotes found, make a new one? 34775 n) New remote 34776 s) Set configuration password 34777 q) Quit config 34778 n/s/q> n 34779 \f[R] 34780 .fi 34781 .IP "2." 3 34782 Enter the name for the configuration 34783 .IP 34784 .nf 34785 \f[C] 34786 name> <YOUR NAME> 34787 \f[R] 34788 .fi 34789 .IP "3." 3 34790 Select \[dq]s3\[dq] storage. 34791 .IP 34792 .nf 34793 \f[C] 34794 Choose a number from below, or type in your own value 34795 [snip] 34796 XX / Amazon S3 Compliant Storage Providers including AWS, ... 34797 \[rs] \[dq]s3\[dq] 34798 [snip] 34799 Storage> s3 34800 \f[R] 34801 .fi 34802 .IP "4." 3 34803 Select IBM COS as the S3 Storage Provider. 34804 .IP 34805 .nf 34806 \f[C] 34807 Choose the S3 provider. 34808 Choose a number from below, or type in your own value 34809 1 / Choose this option to configure Storage to AWS S3 34810 \[rs] \[dq]AWS\[dq] 34811 2 / Choose this option to configure Storage to Ceph Systems 34812 \[rs] \[dq]Ceph\[dq] 34813 3 / Choose this option to configure Storage to Dreamhost 34814 \[rs] \[dq]Dreamhost\[dq] 34815 4 / Choose this option to the configure Storage to IBM COS S3 34816 \[rs] \[dq]IBMCOS\[dq] 34817 5 / Choose this option to the configure Storage to Minio 34818 \[rs] \[dq]Minio\[dq] 34819 Provider>4 34820 \f[R] 34821 .fi 34822 .IP "5." 3 34823 Enter the Access Key and Secret. 34824 .IP 34825 .nf 34826 \f[C] 34827 AWS Access Key ID - leave blank for anonymous access or runtime credentials. 34828 access_key_id> <> 34829 AWS Secret Access Key (password) - leave blank for anonymous access or runtime credentials. 34830 secret_access_key> <> 34831 \f[R] 34832 .fi 34833 .IP "6." 3 34834 Specify the endpoint for IBM COS. 34835 For Public IBM COS, choose from the option below. 34836 For On Premise IBM COS, enter an endpoint address. 34837 .IP 34838 .nf 34839 \f[C] 34840 Endpoint for IBM COS S3 API. 34841 Specify if using an IBM COS On Premise. 34842 Choose a number from below, or type in your own value 34843 1 / US Cross Region Endpoint 34844 \[rs] \[dq]s3-api.us-geo.objectstorage.softlayer.net\[dq] 34845 2 / US Cross Region Dallas Endpoint 34846 \[rs] \[dq]s3-api.dal.us-geo.objectstorage.softlayer.net\[dq] 34847 3 / US Cross Region Washington DC Endpoint 34848 \[rs] \[dq]s3-api.wdc-us-geo.objectstorage.softlayer.net\[dq] 34849 4 / US Cross Region San Jose Endpoint 34850 \[rs] \[dq]s3-api.sjc-us-geo.objectstorage.softlayer.net\[dq] 34851 5 / US Cross Region Private Endpoint 34852 \[rs] \[dq]s3-api.us-geo.objectstorage.service.networklayer.com\[dq] 34853 6 / US Cross Region Dallas Private Endpoint 34854 \[rs] \[dq]s3-api.dal-us-geo.objectstorage.service.networklayer.com\[dq] 34855 7 / US Cross Region Washington DC Private Endpoint 34856 \[rs] \[dq]s3-api.wdc-us-geo.objectstorage.service.networklayer.com\[dq] 34857 8 / US Cross Region San Jose Private Endpoint 34858 \[rs] \[dq]s3-api.sjc-us-geo.objectstorage.service.networklayer.com\[dq] 34859 9 / US Region East Endpoint 34860 \[rs] \[dq]s3.us-east.objectstorage.softlayer.net\[dq] 34861 10 / US Region East Private Endpoint 34862 \[rs] \[dq]s3.us-east.objectstorage.service.networklayer.com\[dq] 34863 11 / US Region South Endpoint 34864 [snip] 34865 34 / Toronto Single Site Private Endpoint 34866 \[rs] \[dq]s3.tor01.objectstorage.service.networklayer.com\[dq] 34867 endpoint>1 34868 \f[R] 34869 .fi 34870 .IP "7." 3 34871 Specify a IBM COS Location Constraint. 34872 The location constraint must match endpoint when using IBM Cloud Public. 34873 For on-prem COS, do not make a selection from this list, hit enter 34874 .IP 34875 .nf 34876 \f[C] 34877 1 / US Cross Region Standard 34878 \[rs] \[dq]us-standard\[dq] 34879 2 / US Cross Region Vault 34880 \[rs] \[dq]us-vault\[dq] 34881 3 / US Cross Region Cold 34882 \[rs] \[dq]us-cold\[dq] 34883 4 / US Cross Region Flex 34884 \[rs] \[dq]us-flex\[dq] 34885 5 / US East Region Standard 34886 \[rs] \[dq]us-east-standard\[dq] 34887 6 / US East Region Vault 34888 \[rs] \[dq]us-east-vault\[dq] 34889 7 / US East Region Cold 34890 \[rs] \[dq]us-east-cold\[dq] 34891 8 / US East Region Flex 34892 \[rs] \[dq]us-east-flex\[dq] 34893 9 / US South Region Standard 34894 \[rs] \[dq]us-south-standard\[dq] 34895 10 / US South Region Vault 34896 \[rs] \[dq]us-south-vault\[dq] 34897 [snip] 34898 32 / Toronto Flex 34899 \[rs] \[dq]tor01-flex\[dq] 34900 location_constraint>1 34901 \f[R] 34902 .fi 34903 .IP "9." 3 34904 Specify a canned ACL. 34905 IBM Cloud (Storage) supports \[dq]public-read\[dq] and 34906 \[dq]private\[dq]. 34907 IBM Cloud(Infra) supports all the canned ACLs. 34908 On-Premise COS supports all the canned ACLs. 34909 .IP 34910 .nf 34911 \f[C] 34912 Canned ACL used when creating buckets and/or storing objects in S3. 34913 For more info visit https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl 34914 Choose a number from below, or type in your own value 34915 1 / Owner gets FULL_CONTROL. No one else has access rights (default). This acl is available on IBM Cloud (Infra), IBM Cloud (Storage), On-Premise COS 34916 \[rs] \[dq]private\[dq] 34917 2 / Owner gets FULL_CONTROL. The AllUsers group gets READ access. This acl is available on IBM Cloud (Infra), IBM Cloud (Storage), On-Premise IBM COS 34918 \[rs] \[dq]public-read\[dq] 34919 3 / Owner gets FULL_CONTROL. The AllUsers group gets READ and WRITE access. This acl is available on IBM Cloud (Infra), On-Premise IBM COS 34920 \[rs] \[dq]public-read-write\[dq] 34921 4 / Owner gets FULL_CONTROL. The AuthenticatedUsers group gets READ access. Not supported on Buckets. This acl is available on IBM Cloud (Infra) and On-Premise IBM COS 34922 \[rs] \[dq]authenticated-read\[dq] 34923 acl> 1 34924 \f[R] 34925 .fi 34926 .IP "12." 4 34927 Review the displayed configuration and accept to save the 34928 \[dq]remote\[dq] then quit. 34929 The config file should look like this 34930 .IP 34931 .nf 34932 \f[C] 34933 [xxx] 34934 type = s3 34935 Provider = IBMCOS 34936 access_key_id = xxx 34937 secret_access_key = yyy 34938 endpoint = s3-api.us-geo.objectstorage.softlayer.net 34939 location_constraint = us-standard 34940 acl = private 34941 \f[R] 34942 .fi 34943 .IP "13." 4 34944 Execute rclone commands 34945 .IP 34946 .nf 34947 \f[C] 34948 1) Create a bucket. 34949 rclone mkdir IBM-COS-XREGION:newbucket 34950 2) List available buckets. 34951 rclone lsd IBM-COS-XREGION: 34952 -1 2017-11-08 21:16:22 -1 test 34953 -1 2018-02-14 20:16:39 -1 newbucket 34954 3) List contents of a bucket. 34955 rclone ls IBM-COS-XREGION:newbucket 34956 18685952 test.exe 34957 4) Copy a file from local to remote. 34958 rclone copy /Users/file.txt IBM-COS-XREGION:newbucket 34959 5) Copy a file from remote to local. 34960 rclone copy IBM-COS-XREGION:newbucket/file.txt . 34961 6) Delete a file on remote. 34962 rclone delete IBM-COS-XREGION:newbucket/file.txt 34963 \f[R] 34964 .fi 34965 .SS IDrive e2 34966 .PP 34967 Here is an example of making an IDrive e2 (https://www.idrive.com/e2/) 34968 configuration. 34969 First run: 34970 .IP 34971 .nf 34972 \f[C] 34973 rclone config 34974 \f[R] 34975 .fi 34976 .PP 34977 This will guide you through an interactive setup process. 34978 .IP 34979 .nf 34980 \f[C] 34981 No remotes found, make a new one? 34982 n) New remote 34983 s) Set configuration password 34984 q) Quit config 34985 n/s/q> n 34986 34987 Enter name for new remote. 34988 name> e2 34989 34990 Option Storage. 34991 Type of storage to configure. 34992 Choose a number from below, or type in your own value. 34993 [snip] 34994 XX / Amazon S3 Compliant Storage Providers including AWS, ... 34995 \[rs] (s3) 34996 [snip] 34997 Storage> s3 34998 34999 Option provider. 35000 Choose your S3 provider. 35001 Choose a number from below, or type in your own value. 35002 Press Enter to leave empty. 35003 [snip] 35004 XX / IDrive e2 35005 \[rs] (IDrive) 35006 [snip] 35007 provider> IDrive 35008 35009 Option env_auth. 35010 Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars). 35011 Only applies if access_key_id and secret_access_key is blank. 35012 Choose a number from below, or type in your own boolean value (true or false). 35013 Press Enter for the default (false). 35014 1 / Enter AWS credentials in the next step. 35015 \[rs] (false) 35016 2 / Get AWS credentials from the environment (env vars or IAM). 35017 \[rs] (true) 35018 env_auth> 35019 35020 Option access_key_id. 35021 AWS Access Key ID. 35022 Leave blank for anonymous access or runtime credentials. 35023 Enter a value. Press Enter to leave empty. 35024 access_key_id> YOUR_ACCESS_KEY 35025 35026 Option secret_access_key. 35027 AWS Secret Access Key (password). 35028 Leave blank for anonymous access or runtime credentials. 35029 Enter a value. Press Enter to leave empty. 35030 secret_access_key> YOUR_SECRET_KEY 35031 35032 Option acl. 35033 Canned ACL used when creating buckets and storing or copying objects. 35034 This ACL is used for creating objects and if bucket_acl isn\[aq]t set, for creating buckets too. 35035 For more info visit https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl 35036 Note that this ACL is applied when server-side copying objects as S3 35037 doesn\[aq]t copy the ACL from the source but rather writes a fresh one. 35038 Choose a number from below, or type in your own value. 35039 Press Enter to leave empty. 35040 / Owner gets FULL_CONTROL. 35041 1 | No one else has access rights (default). 35042 \[rs] (private) 35043 / Owner gets FULL_CONTROL. 35044 2 | The AllUsers group gets READ access. 35045 \[rs] (public-read) 35046 / Owner gets FULL_CONTROL. 35047 3 | The AllUsers group gets READ and WRITE access. 35048 | Granting this on a bucket is generally not recommended. 35049 \[rs] (public-read-write) 35050 / Owner gets FULL_CONTROL. 35051 4 | The AuthenticatedUsers group gets READ access. 35052 \[rs] (authenticated-read) 35053 / Object owner gets FULL_CONTROL. 35054 5 | Bucket owner gets READ access. 35055 | If you specify this canned ACL when creating a bucket, Amazon S3 ignores it. 35056 \[rs] (bucket-owner-read) 35057 / Both the object owner and the bucket owner get FULL_CONTROL over the object. 35058 6 | If you specify this canned ACL when creating a bucket, Amazon S3 ignores it. 35059 \[rs] (bucket-owner-full-control) 35060 acl> 35061 35062 Edit advanced config? 35063 y) Yes 35064 n) No (default) 35065 y/n> 35066 35067 Configuration complete. 35068 Options: 35069 - type: s3 35070 - provider: IDrive 35071 - access_key_id: YOUR_ACCESS_KEY 35072 - secret_access_key: YOUR_SECRET_KEY 35073 - endpoint: q9d9.la12.idrivee2-5.com 35074 Keep this \[dq]e2\[dq] remote? 35075 y) Yes this is OK (default) 35076 e) Edit this remote 35077 d) Delete this remote 35078 y/e/d> y 35079 \f[R] 35080 .fi 35081 .SS IONOS Cloud 35082 .PP 35083 IONOS S3 Object Storage (https://cloud.ionos.com/storage/object-storage) 35084 is a service offered by IONOS for storing and accessing unstructured 35085 data. 35086 To connect to the service, you will need an access key and a secret key. 35087 These can be found in the Data Center Designer (https://dcd.ionos.com/), 35088 by selecting \f[B]Manager resources\f[R] > \f[B]Object Storage Key 35089 Manager\f[R]. 35090 .PP 35091 Here is an example of a configuration. 35092 First, run \f[C]rclone config\f[R]. 35093 This will walk you through an interactive setup process. 35094 Type \f[C]n\f[R] to add the new remote, and then enter a name: 35095 .IP 35096 .nf 35097 \f[C] 35098 Enter name for new remote. 35099 name> ionos-fra 35100 \f[R] 35101 .fi 35102 .PP 35103 Type \f[C]s3\f[R] to choose the connection type: 35104 .IP 35105 .nf 35106 \f[C] 35107 Option Storage. 35108 Type of storage to configure. 35109 Choose a number from below, or type in your own value. 35110 [snip] 35111 XX / Amazon S3 Compliant Storage Providers including AWS, ... 35112 \[rs] (s3) 35113 [snip] 35114 Storage> s3 35115 \f[R] 35116 .fi 35117 .PP 35118 Type \f[C]IONOS\f[R]: 35119 .IP 35120 .nf 35121 \f[C] 35122 Option provider. 35123 Choose your S3 provider. 35124 Choose a number from below, or type in your own value. 35125 Press Enter to leave empty. 35126 [snip] 35127 XX / IONOS Cloud 35128 \[rs] (IONOS) 35129 [snip] 35130 provider> IONOS 35131 \f[R] 35132 .fi 35133 .PP 35134 Press Enter to choose the default option 35135 \f[C]Enter AWS credentials in the next step\f[R]: 35136 .IP 35137 .nf 35138 \f[C] 35139 Option env_auth. 35140 Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars). 35141 Only applies if access_key_id and secret_access_key is blank. 35142 Choose a number from below, or type in your own boolean value (true or false). 35143 Press Enter for the default (false). 35144 1 / Enter AWS credentials in the next step. 35145 \[rs] (false) 35146 2 / Get AWS credentials from the environment (env vars or IAM). 35147 \[rs] (true) 35148 env_auth> 35149 \f[R] 35150 .fi 35151 .PP 35152 Enter your Access Key and Secret key. 35153 These can be retrieved in the Data Center 35154 Designer (https://dcd.ionos.com/), click on the menu \[lq]Manager 35155 resources\[rq] / \[dq]Object Storage Key Manager\[dq]. 35156 .IP 35157 .nf 35158 \f[C] 35159 Option access_key_id. 35160 AWS Access Key ID. 35161 Leave blank for anonymous access or runtime credentials. 35162 Enter a value. Press Enter to leave empty. 35163 access_key_id> YOUR_ACCESS_KEY 35164 35165 Option secret_access_key. 35166 AWS Secret Access Key (password). 35167 Leave blank for anonymous access or runtime credentials. 35168 Enter a value. Press Enter to leave empty. 35169 secret_access_key> YOUR_SECRET_KEY 35170 \f[R] 35171 .fi 35172 .PP 35173 Choose the region where your bucket is located: 35174 .IP 35175 .nf 35176 \f[C] 35177 Option region. 35178 Region where your bucket will be created and your data stored. 35179 Choose a number from below, or type in your own value. 35180 Press Enter to leave empty. 35181 1 / Frankfurt, Germany 35182 \[rs] (de) 35183 2 / Berlin, Germany 35184 \[rs] (eu-central-2) 35185 3 / Logrono, Spain 35186 \[rs] (eu-south-2) 35187 region> 2 35188 \f[R] 35189 .fi 35190 .PP 35191 Choose the endpoint from the same region: 35192 .IP 35193 .nf 35194 \f[C] 35195 Option endpoint. 35196 Endpoint for IONOS S3 Object Storage. 35197 Specify the endpoint from the same region. 35198 Choose a number from below, or type in your own value. 35199 Press Enter to leave empty. 35200 1 / Frankfurt, Germany 35201 \[rs] (s3-eu-central-1.ionoscloud.com) 35202 2 / Berlin, Germany 35203 \[rs] (s3-eu-central-2.ionoscloud.com) 35204 3 / Logrono, Spain 35205 \[rs] (s3-eu-south-2.ionoscloud.com) 35206 endpoint> 1 35207 \f[R] 35208 .fi 35209 .PP 35210 Press Enter to choose the default option or choose the desired ACL 35211 setting: 35212 .IP 35213 .nf 35214 \f[C] 35215 Option acl. 35216 Canned ACL used when creating buckets and storing or copying objects. 35217 This ACL is used for creating objects and if bucket_acl isn\[aq]t set, for creating buckets too. 35218 For more info visit https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl 35219 Note that this ACL is applied when server-side copying objects as S3 35220 doesn\[aq]t copy the ACL from the source but rather writes a fresh one. 35221 Choose a number from below, or type in your own value. 35222 Press Enter to leave empty. 35223 / Owner gets FULL_CONTROL. 35224 1 | No one else has access rights (default). 35225 \[rs] (private) 35226 / Owner gets FULL_CONTROL. 35227 [snip] 35228 acl> 35229 \f[R] 35230 .fi 35231 .PP 35232 Press Enter to skip the advanced config: 35233 .IP 35234 .nf 35235 \f[C] 35236 Edit advanced config? 35237 y) Yes 35238 n) No (default) 35239 y/n> 35240 \f[R] 35241 .fi 35242 .PP 35243 Press Enter to save the configuration, and then \f[C]q\f[R] to quit the 35244 configuration process: 35245 .IP 35246 .nf 35247 \f[C] 35248 Configuration complete. 35249 Options: 35250 - type: s3 35251 - provider: IONOS 35252 - access_key_id: YOUR_ACCESS_KEY 35253 - secret_access_key: YOUR_SECRET_KEY 35254 - endpoint: s3-eu-central-1.ionoscloud.com 35255 Keep this \[dq]ionos-fra\[dq] remote? 35256 y) Yes this is OK (default) 35257 e) Edit this remote 35258 d) Delete this remote 35259 y/e/d> y 35260 \f[R] 35261 .fi 35262 .PP 35263 Done! Now you can try some commands (for macOS, use \f[C]./rclone\f[R] 35264 instead of \f[C]rclone\f[R]). 35265 .IP "1)" 3 35266 Create a bucket (the name must be unique within the whole IONOS S3) 35267 .IP 35268 .nf 35269 \f[C] 35270 rclone mkdir ionos-fra:my-bucket 35271 \f[R] 35272 .fi 35273 .IP "2)" 3 35274 List available buckets 35275 .IP 35276 .nf 35277 \f[C] 35278 rclone lsd ionos-fra: 35279 \f[R] 35280 .fi 35281 .IP "4)" 3 35282 Copy a file from local to remote 35283 .IP 35284 .nf 35285 \f[C] 35286 rclone copy /Users/file.txt ionos-fra:my-bucket 35287 \f[R] 35288 .fi 35289 .IP "3)" 3 35290 List contents of a bucket 35291 .IP 35292 .nf 35293 \f[C] 35294 rclone ls ionos-fra:my-bucket 35295 \f[R] 35296 .fi 35297 .IP "5)" 3 35298 Copy a file from remote to local 35299 .IP 35300 .nf 35301 \f[C] 35302 rclone copy ionos-fra:my-bucket/file.txt 35303 \f[R] 35304 .fi 35305 .SS Minio 35306 .PP 35307 Minio (https://minio.io/) is an object storage server built for cloud 35308 application developers and devops. 35309 .PP 35310 It is very easy to install and provides an S3 compatible server which 35311 can be used by rclone. 35312 .PP 35313 To use it, install Minio following the instructions 35314 here (https://docs.minio.io/docs/minio-quickstart-guide). 35315 .PP 35316 When it configures itself Minio will print something like this 35317 .IP 35318 .nf 35319 \f[C] 35320 Endpoint: http://192.168.1.106:9000 http://172.23.0.1:9000 35321 AccessKey: USWUXHGYZQYFYFFIT3RE 35322 SecretKey: MOJRH0mkL1IPauahWITSVvyDrQbEEIwljvmxdq03 35323 Region: us-east-1 35324 SQS ARNs: arn:minio:sqs:us-east-1:1:redis arn:minio:sqs:us-east-1:2:redis 35325 35326 Browser Access: 35327 http://192.168.1.106:9000 http://172.23.0.1:9000 35328 35329 Command-line Access: https://docs.minio.io/docs/minio-client-quickstart-guide 35330 $ mc config host add myminio http://192.168.1.106:9000 USWUXHGYZQYFYFFIT3RE MOJRH0mkL1IPauahWITSVvyDrQbEEIwljvmxdq03 35331 35332 Object API (Amazon S3 compatible): 35333 Go: https://docs.minio.io/docs/golang-client-quickstart-guide 35334 Java: https://docs.minio.io/docs/java-client-quickstart-guide 35335 Python: https://docs.minio.io/docs/python-client-quickstart-guide 35336 JavaScript: https://docs.minio.io/docs/javascript-client-quickstart-guide 35337 .NET: https://docs.minio.io/docs/dotnet-client-quickstart-guide 35338 35339 Drive Capacity: 26 GiB Free, 165 GiB Total 35340 \f[R] 35341 .fi 35342 .PP 35343 These details need to go into \f[C]rclone config\f[R] like this. 35344 Note that it is important to put the region in as stated above. 35345 .IP 35346 .nf 35347 \f[C] 35348 env_auth> 1 35349 access_key_id> USWUXHGYZQYFYFFIT3RE 35350 secret_access_key> MOJRH0mkL1IPauahWITSVvyDrQbEEIwljvmxdq03 35351 region> us-east-1 35352 endpoint> http://192.168.1.106:9000 35353 location_constraint> 35354 server_side_encryption> 35355 \f[R] 35356 .fi 35357 .PP 35358 Which makes the config file look like this 35359 .IP 35360 .nf 35361 \f[C] 35362 [minio] 35363 type = s3 35364 provider = Minio 35365 env_auth = false 35366 access_key_id = USWUXHGYZQYFYFFIT3RE 35367 secret_access_key = MOJRH0mkL1IPauahWITSVvyDrQbEEIwljvmxdq03 35368 region = us-east-1 35369 endpoint = http://192.168.1.106:9000 35370 location_constraint = 35371 server_side_encryption = 35372 \f[R] 35373 .fi 35374 .PP 35375 So once set up, for example, to copy files into a bucket 35376 .IP 35377 .nf 35378 \f[C] 35379 rclone copy /path/to/files minio:bucket 35380 \f[R] 35381 .fi 35382 .SS Qiniu Cloud Object Storage (Kodo) 35383 .PP 35384 Qiniu Cloud Object Storage 35385 (Kodo) (https://www.qiniu.com/en/products/kodo), a completely 35386 independent-researched core technology which is proven by repeated 35387 customer experience has occupied absolute leading market leader 35388 position. 35389 Kodo can be widely applied to mass data management. 35390 .PP 35391 To configure access to Qiniu Kodo, follow the steps below: 35392 .IP "1." 3 35393 Run \f[C]rclone config\f[R] and select \f[C]n\f[R] for a new remote. 35394 .IP 35395 .nf 35396 \f[C] 35397 rclone config 35398 No remotes found, make a new one? 35399 n) New remote 35400 s) Set configuration password 35401 q) Quit config 35402 n/s/q> n 35403 \f[R] 35404 .fi 35405 .IP "2." 3 35406 Give the name of the configuration. 35407 For example, name it \[aq]qiniu\[aq]. 35408 .IP 35409 .nf 35410 \f[C] 35411 name> qiniu 35412 \f[R] 35413 .fi 35414 .IP "3." 3 35415 Select \f[C]s3\f[R] storage. 35416 .IP 35417 .nf 35418 \f[C] 35419 Choose a number from below, or type in your own value 35420 [snip] 35421 XX / Amazon S3 Compliant Storage Providers including AWS, ... 35422 \[rs] (s3) 35423 [snip] 35424 Storage> s3 35425 \f[R] 35426 .fi 35427 .IP "4." 3 35428 Select \f[C]Qiniu\f[R] provider. 35429 .IP 35430 .nf 35431 \f[C] 35432 Choose a number from below, or type in your own value 35433 1 / Amazon Web Services (AWS) S3 35434 \[rs] \[dq]AWS\[dq] 35435 [snip] 35436 22 / Qiniu Object Storage (Kodo) 35437 \[rs] (Qiniu) 35438 [snip] 35439 provider> Qiniu 35440 \f[R] 35441 .fi 35442 .IP "5." 3 35443 Enter your SecretId and SecretKey of Qiniu Kodo. 35444 .IP 35445 .nf 35446 \f[C] 35447 Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars). 35448 Only applies if access_key_id and secret_access_key is blank. 35449 Enter a boolean value (true or false). Press Enter for the default (\[dq]false\[dq]). 35450 Choose a number from below, or type in your own value 35451 1 / Enter AWS credentials in the next step 35452 \[rs] \[dq]false\[dq] 35453 2 / Get AWS credentials from the environment (env vars or IAM) 35454 \[rs] \[dq]true\[dq] 35455 env_auth> 1 35456 AWS Access Key ID. 35457 Leave blank for anonymous access or runtime credentials. 35458 Enter a string value. Press Enter for the default (\[dq]\[dq]). 35459 access_key_id> AKIDxxxxxxxxxx 35460 AWS Secret Access Key (password) 35461 Leave blank for anonymous access or runtime credentials. 35462 Enter a string value. Press Enter for the default (\[dq]\[dq]). 35463 secret_access_key> xxxxxxxxxxx 35464 \f[R] 35465 .fi 35466 .IP "6." 3 35467 Select endpoint for Qiniu Kodo. 35468 This is the standard endpoint for different region. 35469 .IP 35470 .nf 35471 \f[C] 35472 / The default endpoint - a good choice if you are unsure. 35473 1 | East China Region 1. 35474 | Needs location constraint cn-east-1. 35475 \[rs] (cn-east-1) 35476 / East China Region 2. 35477 2 | Needs location constraint cn-east-2. 35478 \[rs] (cn-east-2) 35479 / North China Region 1. 35480 3 | Needs location constraint cn-north-1. 35481 \[rs] (cn-north-1) 35482 / South China Region 1. 35483 4 | Needs location constraint cn-south-1. 35484 \[rs] (cn-south-1) 35485 / North America Region. 35486 5 | Needs location constraint us-north-1. 35487 \[rs] (us-north-1) 35488 / Southeast Asia Region 1. 35489 6 | Needs location constraint ap-southeast-1. 35490 \[rs] (ap-southeast-1) 35491 / Northeast Asia Region 1. 35492 7 | Needs location constraint ap-northeast-1. 35493 \[rs] (ap-northeast-1) 35494 [snip] 35495 endpoint> 1 35496 35497 Option endpoint. 35498 Endpoint for Qiniu Object Storage. 35499 Choose a number from below, or type in your own value. 35500 Press Enter to leave empty. 35501 1 / East China Endpoint 1 35502 \[rs] (s3-cn-east-1.qiniucs.com) 35503 2 / East China Endpoint 2 35504 \[rs] (s3-cn-east-2.qiniucs.com) 35505 3 / North China Endpoint 1 35506 \[rs] (s3-cn-north-1.qiniucs.com) 35507 4 / South China Endpoint 1 35508 \[rs] (s3-cn-south-1.qiniucs.com) 35509 5 / North America Endpoint 1 35510 \[rs] (s3-us-north-1.qiniucs.com) 35511 6 / Southeast Asia Endpoint 1 35512 \[rs] (s3-ap-southeast-1.qiniucs.com) 35513 7 / Northeast Asia Endpoint 1 35514 \[rs] (s3-ap-northeast-1.qiniucs.com) 35515 endpoint> 1 35516 35517 Option location_constraint. 35518 Location constraint - must be set to match the Region. 35519 Used when creating buckets only. 35520 Choose a number from below, or type in your own value. 35521 Press Enter to leave empty. 35522 1 / East China Region 1 35523 \[rs] (cn-east-1) 35524 2 / East China Region 2 35525 \[rs] (cn-east-2) 35526 3 / North China Region 1 35527 \[rs] (cn-north-1) 35528 4 / South China Region 1 35529 \[rs] (cn-south-1) 35530 5 / North America Region 1 35531 \[rs] (us-north-1) 35532 6 / Southeast Asia Region 1 35533 \[rs] (ap-southeast-1) 35534 7 / Northeast Asia Region 1 35535 \[rs] (ap-northeast-1) 35536 location_constraint> 1 35537 \f[R] 35538 .fi 35539 .IP "7." 3 35540 Choose acl and storage class. 35541 .IP 35542 .nf 35543 \f[C] 35544 Note that this ACL is applied when server-side copying objects as S3 35545 doesn\[aq]t copy the ACL from the source but rather writes a fresh one. 35546 Enter a string value. Press Enter for the default (\[dq]\[dq]). 35547 Choose a number from below, or type in your own value 35548 / Owner gets FULL_CONTROL. 35549 1 | No one else has access rights (default). 35550 \[rs] (private) 35551 / Owner gets FULL_CONTROL. 35552 2 | The AllUsers group gets READ access. 35553 \[rs] (public-read) 35554 [snip] 35555 acl> 2 35556 The storage class to use when storing new objects in Tencent COS. 35557 Enter a string value. Press Enter for the default (\[dq]\[dq]). 35558 Choose a number from below, or type in your own value 35559 1 / Standard storage class 35560 \[rs] (STANDARD) 35561 2 / Infrequent access storage mode 35562 \[rs] (LINE) 35563 3 / Archive storage mode 35564 \[rs] (GLACIER) 35565 4 / Deep archive storage mode 35566 \[rs] (DEEP_ARCHIVE) 35567 [snip] 35568 storage_class> 1 35569 Edit advanced config? (y/n) 35570 y) Yes 35571 n) No (default) 35572 y/n> n 35573 Remote config 35574 -------------------- 35575 [qiniu] 35576 - type: s3 35577 - provider: Qiniu 35578 - access_key_id: xxx 35579 - secret_access_key: xxx 35580 - region: cn-east-1 35581 - endpoint: s3-cn-east-1.qiniucs.com 35582 - location_constraint: cn-east-1 35583 - acl: public-read 35584 - storage_class: STANDARD 35585 -------------------- 35586 y) Yes this is OK (default) 35587 e) Edit this remote 35588 d) Delete this remote 35589 y/e/d> y 35590 Current remotes: 35591 35592 Name Type 35593 ==== ==== 35594 qiniu s3 35595 \f[R] 35596 .fi 35597 .SS RackCorp 35598 .PP 35599 RackCorp Object Storage (https://www.rackcorp.com/storage/s3storage) is 35600 an S3 compatible object storage platform from your friendly cloud 35601 provider RackCorp. 35602 The service is fast, reliable, well priced and located in many strategic 35603 locations unserviced by others, to ensure you can maintain data 35604 sovereignty. 35605 .PP 35606 Before you can use RackCorp Object Storage, you\[aq]ll need to \[dq]sign 35607 up (https://www.rackcorp.com/signup)\[dq] for an account on our 35608 \[dq]portal (https://portal.rackcorp.com)\[dq]. 35609 Next you can create an \f[C]access key\f[R], a \f[C]secret key\f[R] and 35610 \f[C]buckets\f[R], in your location of choice with ease. 35611 These details are required for the next steps of configuration, when 35612 \f[C]rclone config\f[R] asks for your \f[C]access_key_id\f[R] and 35613 \f[C]secret_access_key\f[R]. 35614 .PP 35615 Your config should end up looking a bit like this: 35616 .IP 35617 .nf 35618 \f[C] 35619 [RCS3-demo-config] 35620 type = s3 35621 provider = RackCorp 35622 env_auth = true 35623 access_key_id = YOURACCESSKEY 35624 secret_access_key = YOURSECRETACCESSKEY 35625 region = au-nsw 35626 endpoint = s3.rackcorp.com 35627 location_constraint = au-nsw 35628 \f[R] 35629 .fi 35630 .SS Rclone Serve S3 35631 .PP 35632 Rclone can serve any remote over the S3 protocol. 35633 For details see the rclone serve 35634 s3 (https://rclone.org/commands/rclone_serve_http/) documentation. 35635 .PP 35636 For example, to serve \f[C]remote:path\f[R] over s3, run the server like 35637 this: 35638 .IP 35639 .nf 35640 \f[C] 35641 rclone serve s3 --auth-key ACCESS_KEY_ID,SECRET_ACCESS_KEY remote:path 35642 \f[R] 35643 .fi 35644 .PP 35645 This will be compatible with an rclone remote which is defined like 35646 this: 35647 .IP 35648 .nf 35649 \f[C] 35650 [serves3] 35651 type = s3 35652 provider = Rclone 35653 endpoint = http://127.0.0.1:8080/ 35654 access_key_id = ACCESS_KEY_ID 35655 secret_access_key = SECRET_ACCESS_KEY 35656 use_multipart_uploads = false 35657 \f[R] 35658 .fi 35659 .PP 35660 Note that setting \f[C]disable_multipart_uploads = true\f[R] is to work 35661 around a bug (https://rclone.org/commands/rclone_serve_http/#bugs) which 35662 will be fixed in due course. 35663 .SS Scaleway 35664 .PP 35665 Scaleway (https://www.scaleway.com/object-storage/) The Object Storage 35666 platform allows you to store anything from backups, logs and web assets 35667 to documents and photos. 35668 Files can be dropped from the Scaleway console or transferred through 35669 our API and CLI or using any S3-compatible tool. 35670 .PP 35671 Scaleway provides an S3 interface which can be configured for use with 35672 rclone like this: 35673 .IP 35674 .nf 35675 \f[C] 35676 [scaleway] 35677 type = s3 35678 provider = Scaleway 35679 env_auth = false 35680 endpoint = s3.nl-ams.scw.cloud 35681 access_key_id = SCWXXXXXXXXXXXXXX 35682 secret_access_key = 1111111-2222-3333-44444-55555555555555 35683 region = nl-ams 35684 location_constraint = 35685 acl = private 35686 server_side_encryption = 35687 storage_class = 35688 \f[R] 35689 .fi 35690 .PP 35691 C14 Cold Storage (https://www.online.net/en/storage/c14-cold-storage) is 35692 the low-cost S3 Glacier alternative from Scaleway and it works the same 35693 way as on S3 by accepting the \[dq]GLACIER\[dq] \f[C]storage_class\f[R]. 35694 So you can configure your remote with the 35695 \f[C]storage_class = GLACIER\f[R] option to upload directly to C14. 35696 Don\[aq]t forget that in this state you can\[aq]t read files back after, 35697 you will need to restore them to \[dq]STANDARD\[dq] storage_class first 35698 before being able to read them (see \[dq]restore\[dq] section above) 35699 .SS Seagate Lyve Cloud 35700 .PP 35701 Seagate Lyve 35702 Cloud (https://www.seagate.com/gb/en/services/cloud/storage/) is an S3 35703 compatible object storage platform from Seagate (https://seagate.com/) 35704 intended for enterprise use. 35705 .PP 35706 Here is a config run through for a remote called \f[C]remote\f[R] - you 35707 may choose a different name of course. 35708 Note that to create an access key and secret key you will need to create 35709 a service account first. 35710 .IP 35711 .nf 35712 \f[C] 35713 $ rclone config 35714 No remotes found, make a new one? 35715 n) New remote 35716 s) Set configuration password 35717 q) Quit config 35718 n/s/q> n 35719 name> remote 35720 \f[R] 35721 .fi 35722 .PP 35723 Choose \f[C]s3\f[R] backend 35724 .IP 35725 .nf 35726 \f[C] 35727 Type of storage to configure. 35728 Choose a number from below, or type in your own value. 35729 [snip] 35730 XX / Amazon S3 Compliant Storage Providers including AWS, ... 35731 \[rs] (s3) 35732 [snip] 35733 Storage> s3 35734 \f[R] 35735 .fi 35736 .PP 35737 Choose \f[C]LyveCloud\f[R] as S3 provider 35738 .IP 35739 .nf 35740 \f[C] 35741 Choose your S3 provider. 35742 Choose a number from below, or type in your own value. 35743 Press Enter to leave empty. 35744 [snip] 35745 XX / Seagate Lyve Cloud 35746 \[rs] (LyveCloud) 35747 [snip] 35748 provider> LyveCloud 35749 \f[R] 35750 .fi 35751 .PP 35752 Take the default (just press enter) to enter access key and secret in 35753 the config file. 35754 .IP 35755 .nf 35756 \f[C] 35757 Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars). 35758 Only applies if access_key_id and secret_access_key is blank. 35759 Choose a number from below, or type in your own boolean value (true or false). 35760 Press Enter for the default (false). 35761 1 / Enter AWS credentials in the next step. 35762 \[rs] (false) 35763 2 / Get AWS credentials from the environment (env vars or IAM). 35764 \[rs] (true) 35765 env_auth> 35766 \f[R] 35767 .fi 35768 .IP 35769 .nf 35770 \f[C] 35771 AWS Access Key ID. 35772 Leave blank for anonymous access or runtime credentials. 35773 Enter a value. Press Enter to leave empty. 35774 access_key_id> XXX 35775 \f[R] 35776 .fi 35777 .IP 35778 .nf 35779 \f[C] 35780 AWS Secret Access Key (password). 35781 Leave blank for anonymous access or runtime credentials. 35782 Enter a value. Press Enter to leave empty. 35783 secret_access_key> YYY 35784 \f[R] 35785 .fi 35786 .PP 35787 Leave region blank 35788 .IP 35789 .nf 35790 \f[C] 35791 Region to connect to. 35792 Leave blank if you are using an S3 clone and you don\[aq]t have a region. 35793 Choose a number from below, or type in your own value. 35794 Press Enter to leave empty. 35795 / Use this if unsure. 35796 1 | Will use v4 signatures and an empty region. 35797 \[rs] () 35798 / Use this only if v4 signatures don\[aq]t work. 35799 2 | E.g. pre Jewel/v10 CEPH. 35800 \[rs] (other-v2-signature) 35801 region> 35802 \f[R] 35803 .fi 35804 .PP 35805 Choose an endpoint from the list 35806 .IP 35807 .nf 35808 \f[C] 35809 Endpoint for S3 API. 35810 Required when using an S3 clone. 35811 Choose a number from below, or type in your own value. 35812 Press Enter to leave empty. 35813 1 / Seagate Lyve Cloud US East 1 (Virginia) 35814 \[rs] (s3.us-east-1.lyvecloud.seagate.com) 35815 2 / Seagate Lyve Cloud US West 1 (California) 35816 \[rs] (s3.us-west-1.lyvecloud.seagate.com) 35817 3 / Seagate Lyve Cloud AP Southeast 1 (Singapore) 35818 \[rs] (s3.ap-southeast-1.lyvecloud.seagate.com) 35819 endpoint> 1 35820 \f[R] 35821 .fi 35822 .PP 35823 Leave location constraint blank 35824 .IP 35825 .nf 35826 \f[C] 35827 Location constraint - must be set to match the Region. 35828 Leave blank if not sure. Used when creating buckets only. 35829 Enter a value. Press Enter to leave empty. 35830 location_constraint> 35831 \f[R] 35832 .fi 35833 .PP 35834 Choose default ACL (\f[C]private\f[R]). 35835 .IP 35836 .nf 35837 \f[C] 35838 Canned ACL used when creating buckets and storing or copying objects. 35839 This ACL is used for creating objects and if bucket_acl isn\[aq]t set, for creating buckets too. 35840 For more info visit https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl 35841 Note that this ACL is applied when server-side copying objects as S3 35842 doesn\[aq]t copy the ACL from the source but rather writes a fresh one. 35843 Choose a number from below, or type in your own value. 35844 Press Enter to leave empty. 35845 / Owner gets FULL_CONTROL. 35846 1 | No one else has access rights (default). 35847 \[rs] (private) 35848 [snip] 35849 acl> 35850 \f[R] 35851 .fi 35852 .PP 35853 And the config file should end up looking like this: 35854 .IP 35855 .nf 35856 \f[C] 35857 [remote] 35858 type = s3 35859 provider = LyveCloud 35860 access_key_id = XXX 35861 secret_access_key = YYY 35862 endpoint = s3.us-east-1.lyvecloud.seagate.com 35863 \f[R] 35864 .fi 35865 .SS SeaweedFS 35866 .PP 35867 SeaweedFS (https://github.com/chrislusf/seaweedfs/) is a distributed 35868 storage system for blobs, objects, files, and data lake, with O(1) disk 35869 seek and a scalable file metadata store. 35870 It has an S3 compatible object storage interface. 35871 SeaweedFS can also act as a gateway to remote S3 compatible object 35872 store (https://github.com/chrislusf/seaweedfs/wiki/Gateway-to-Remote-Object-Storage) 35873 to cache data and metadata with asynchronous write back, for fast local 35874 speed and minimize access cost. 35875 .PP 35876 Assuming the SeaweedFS are configured with \f[C]weed shell\f[R] as such: 35877 .IP 35878 .nf 35879 \f[C] 35880 > s3.bucket.create -name foo 35881 > s3.configure -access_key=any -secret_key=any -buckets=foo -user=me -actions=Read,Write,List,Tagging,Admin -apply 35882 { 35883 \[dq]identities\[dq]: [ 35884 { 35885 \[dq]name\[dq]: \[dq]me\[dq], 35886 \[dq]credentials\[dq]: [ 35887 { 35888 \[dq]accessKey\[dq]: \[dq]any\[dq], 35889 \[dq]secretKey\[dq]: \[dq]any\[dq] 35890 } 35891 ], 35892 \[dq]actions\[dq]: [ 35893 \[dq]Read:foo\[dq], 35894 \[dq]Write:foo\[dq], 35895 \[dq]List:foo\[dq], 35896 \[dq]Tagging:foo\[dq], 35897 \[dq]Admin:foo\[dq] 35898 ] 35899 } 35900 ] 35901 } 35902 \f[R] 35903 .fi 35904 .PP 35905 To use rclone with SeaweedFS, above configuration should end up with 35906 something like this in your config: 35907 .IP 35908 .nf 35909 \f[C] 35910 [seaweedfs_s3] 35911 type = s3 35912 provider = SeaweedFS 35913 access_key_id = any 35914 secret_access_key = any 35915 endpoint = localhost:8333 35916 \f[R] 35917 .fi 35918 .PP 35919 So once set up, for example to copy files into a bucket 35920 .IP 35921 .nf 35922 \f[C] 35923 rclone copy /path/to/files seaweedfs_s3:foo 35924 \f[R] 35925 .fi 35926 .SS Wasabi 35927 .PP 35928 Wasabi (https://wasabi.com) is a cloud-based object storage service for 35929 a broad range of applications and use cases. 35930 Wasabi is designed for individuals and organizations that require a 35931 high-performance, reliable, and secure data storage infrastructure at 35932 minimal cost. 35933 .PP 35934 Wasabi provides an S3 interface which can be configured for use with 35935 rclone like this. 35936 .IP 35937 .nf 35938 \f[C] 35939 No remotes found, make a new one? 35940 n) New remote 35941 s) Set configuration password 35942 n/s> n 35943 name> wasabi 35944 Type of storage to configure. 35945 Choose a number from below, or type in your own value 35946 [snip] 35947 XX / Amazon S3 (also Dreamhost, Ceph, ChinaMobile, ArvanCloud, Minio, Liara) 35948 \[rs] \[dq]s3\[dq] 35949 [snip] 35950 Storage> s3 35951 Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars). Only applies if access_key_id and secret_access_key is blank. 35952 Choose a number from below, or type in your own value 35953 1 / Enter AWS credentials in the next step 35954 \[rs] \[dq]false\[dq] 35955 2 / Get AWS credentials from the environment (env vars or IAM) 35956 \[rs] \[dq]true\[dq] 35957 env_auth> 1 35958 AWS Access Key ID - leave blank for anonymous access or runtime credentials. 35959 access_key_id> YOURACCESSKEY 35960 AWS Secret Access Key (password) - leave blank for anonymous access or runtime credentials. 35961 secret_access_key> YOURSECRETACCESSKEY 35962 Region to connect to. 35963 Choose a number from below, or type in your own value 35964 / The default endpoint - a good choice if you are unsure. 35965 1 | US Region, Northern Virginia, or Pacific Northwest. 35966 | Leave location constraint empty. 35967 \[rs] \[dq]us-east-1\[dq] 35968 [snip] 35969 region> us-east-1 35970 Endpoint for S3 API. 35971 Leave blank if using AWS to use the default endpoint for the region. 35972 Specify if using an S3 clone such as Ceph. 35973 endpoint> s3.wasabisys.com 35974 Location constraint - must be set to match the Region. Used when creating buckets only. 35975 Choose a number from below, or type in your own value 35976 1 / Empty for US Region, Northern Virginia, or Pacific Northwest. 35977 \[rs] \[dq]\[dq] 35978 [snip] 35979 location_constraint> 35980 Canned ACL used when creating buckets and/or storing objects in S3. 35981 For more info visit https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl 35982 Choose a number from below, or type in your own value 35983 1 / Owner gets FULL_CONTROL. No one else has access rights (default). 35984 \[rs] \[dq]private\[dq] 35985 [snip] 35986 acl> 35987 The server-side encryption algorithm used when storing this object in S3. 35988 Choose a number from below, or type in your own value 35989 1 / None 35990 \[rs] \[dq]\[dq] 35991 2 / AES256 35992 \[rs] \[dq]AES256\[dq] 35993 server_side_encryption> 35994 The storage class to use when storing objects in S3. 35995 Choose a number from below, or type in your own value 35996 1 / Default 35997 \[rs] \[dq]\[dq] 35998 2 / Standard storage class 35999 \[rs] \[dq]STANDARD\[dq] 36000 3 / Reduced redundancy storage class 36001 \[rs] \[dq]REDUCED_REDUNDANCY\[dq] 36002 4 / Standard Infrequent Access storage class 36003 \[rs] \[dq]STANDARD_IA\[dq] 36004 storage_class> 36005 Remote config 36006 -------------------- 36007 [wasabi] 36008 env_auth = false 36009 access_key_id = YOURACCESSKEY 36010 secret_access_key = YOURSECRETACCESSKEY 36011 region = us-east-1 36012 endpoint = s3.wasabisys.com 36013 location_constraint = 36014 acl = 36015 server_side_encryption = 36016 storage_class = 36017 -------------------- 36018 y) Yes this is OK 36019 e) Edit this remote 36020 d) Delete this remote 36021 y/e/d> y 36022 \f[R] 36023 .fi 36024 .PP 36025 This will leave the config file looking like this. 36026 .IP 36027 .nf 36028 \f[C] 36029 [wasabi] 36030 type = s3 36031 provider = Wasabi 36032 env_auth = false 36033 access_key_id = YOURACCESSKEY 36034 secret_access_key = YOURSECRETACCESSKEY 36035 region = 36036 endpoint = s3.wasabisys.com 36037 location_constraint = 36038 acl = 36039 server_side_encryption = 36040 storage_class = 36041 \f[R] 36042 .fi 36043 .SS Alibaba OSS 36044 .PP 36045 Here is an example of making an Alibaba Cloud (Aliyun) 36046 OSS (https://www.alibabacloud.com/product/oss/) configuration. 36047 First run: 36048 .IP 36049 .nf 36050 \f[C] 36051 rclone config 36052 \f[R] 36053 .fi 36054 .PP 36055 This will guide you through an interactive setup process. 36056 .IP 36057 .nf 36058 \f[C] 36059 No remotes found, make a new one? 36060 n) New remote 36061 s) Set configuration password 36062 q) Quit config 36063 n/s/q> n 36064 name> oss 36065 Type of storage to configure. 36066 Enter a string value. Press Enter for the default (\[dq]\[dq]). 36067 Choose a number from below, or type in your own value 36068 [snip] 36069 XX / Amazon S3 Compliant Storage Providers including AWS, ... 36070 \[rs] \[dq]s3\[dq] 36071 [snip] 36072 Storage> s3 36073 Choose your S3 provider. 36074 Enter a string value. Press Enter for the default (\[dq]\[dq]). 36075 Choose a number from below, or type in your own value 36076 1 / Amazon Web Services (AWS) S3 36077 \[rs] \[dq]AWS\[dq] 36078 2 / Alibaba Cloud Object Storage System (OSS) formerly Aliyun 36079 \[rs] \[dq]Alibaba\[dq] 36080 3 / Ceph Object Storage 36081 \[rs] \[dq]Ceph\[dq] 36082 [snip] 36083 provider> Alibaba 36084 Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars). 36085 Only applies if access_key_id and secret_access_key is blank. 36086 Enter a boolean value (true or false). Press Enter for the default (\[dq]false\[dq]). 36087 Choose a number from below, or type in your own value 36088 1 / Enter AWS credentials in the next step 36089 \[rs] \[dq]false\[dq] 36090 2 / Get AWS credentials from the environment (env vars or IAM) 36091 \[rs] \[dq]true\[dq] 36092 env_auth> 1 36093 AWS Access Key ID. 36094 Leave blank for anonymous access or runtime credentials. 36095 Enter a string value. Press Enter for the default (\[dq]\[dq]). 36096 access_key_id> accesskeyid 36097 AWS Secret Access Key (password) 36098 Leave blank for anonymous access or runtime credentials. 36099 Enter a string value. Press Enter for the default (\[dq]\[dq]). 36100 secret_access_key> secretaccesskey 36101 Endpoint for OSS API. 36102 Enter a string value. Press Enter for the default (\[dq]\[dq]). 36103 Choose a number from below, or type in your own value 36104 1 / East China 1 (Hangzhou) 36105 \[rs] \[dq]oss-cn-hangzhou.aliyuncs.com\[dq] 36106 2 / East China 2 (Shanghai) 36107 \[rs] \[dq]oss-cn-shanghai.aliyuncs.com\[dq] 36108 3 / North China 1 (Qingdao) 36109 \[rs] \[dq]oss-cn-qingdao.aliyuncs.com\[dq] 36110 [snip] 36111 endpoint> 1 36112 Canned ACL used when creating buckets and storing or copying objects. 36113 36114 Note that this ACL is applied when server-side copying objects as S3 36115 doesn\[aq]t copy the ACL from the source but rather writes a fresh one. 36116 Enter a string value. Press Enter for the default (\[dq]\[dq]). 36117 Choose a number from below, or type in your own value 36118 1 / Owner gets FULL_CONTROL. No one else has access rights (default). 36119 \[rs] \[dq]private\[dq] 36120 2 / Owner gets FULL_CONTROL. The AllUsers group gets READ access. 36121 \[rs] \[dq]public-read\[dq] 36122 / Owner gets FULL_CONTROL. The AllUsers group gets READ and WRITE access. 36123 [snip] 36124 acl> 1 36125 The storage class to use when storing new objects in OSS. 36126 Enter a string value. Press Enter for the default (\[dq]\[dq]). 36127 Choose a number from below, or type in your own value 36128 1 / Default 36129 \[rs] \[dq]\[dq] 36130 2 / Standard storage class 36131 \[rs] \[dq]STANDARD\[dq] 36132 3 / Archive storage mode. 36133 \[rs] \[dq]GLACIER\[dq] 36134 4 / Infrequent access storage mode. 36135 \[rs] \[dq]STANDARD_IA\[dq] 36136 storage_class> 1 36137 Edit advanced config? (y/n) 36138 y) Yes 36139 n) No 36140 y/n> n 36141 Remote config 36142 -------------------- 36143 [oss] 36144 type = s3 36145 provider = Alibaba 36146 env_auth = false 36147 access_key_id = accesskeyid 36148 secret_access_key = secretaccesskey 36149 endpoint = oss-cn-hangzhou.aliyuncs.com 36150 acl = private 36151 storage_class = Standard 36152 -------------------- 36153 y) Yes this is OK 36154 e) Edit this remote 36155 d) Delete this remote 36156 y/e/d> y 36157 \f[R] 36158 .fi 36159 .SS China Mobile Ecloud Elastic Object Storage (EOS) 36160 .PP 36161 Here is an example of making an China Mobile Ecloud Elastic Object 36162 Storage (EOS) (https:///ecloud.10086.cn/home/product-introduction/eos/) 36163 configuration. 36164 First run: 36165 .IP 36166 .nf 36167 \f[C] 36168 rclone config 36169 \f[R] 36170 .fi 36171 .PP 36172 This will guide you through an interactive setup process. 36173 .IP 36174 .nf 36175 \f[C] 36176 No remotes found, make a new one? 36177 n) New remote 36178 s) Set configuration password 36179 q) Quit config 36180 n/s/q> n 36181 name> ChinaMobile 36182 Option Storage. 36183 Type of storage to configure. 36184 Choose a number from below, or type in your own value. 36185 ... 36186 XX / Amazon S3 Compliant Storage Providers including AWS, ... 36187 \[rs] (s3) 36188 ... 36189 Storage> s3 36190 Option provider. 36191 Choose your S3 provider. 36192 Choose a number from below, or type in your own value. 36193 Press Enter to leave empty. 36194 ... 36195 4 / China Mobile Ecloud Elastic Object Storage (EOS) 36196 \[rs] (ChinaMobile) 36197 ... 36198 provider> ChinaMobile 36199 Option env_auth. 36200 Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars). 36201 Only applies if access_key_id and secret_access_key is blank. 36202 Choose a number from below, or type in your own boolean value (true or false). 36203 Press Enter for the default (false). 36204 1 / Enter AWS credentials in the next step. 36205 \[rs] (false) 36206 2 / Get AWS credentials from the environment (env vars or IAM). 36207 \[rs] (true) 36208 env_auth> 36209 Option access_key_id. 36210 AWS Access Key ID. 36211 Leave blank for anonymous access or runtime credentials. 36212 Enter a value. Press Enter to leave empty. 36213 access_key_id> accesskeyid 36214 Option secret_access_key. 36215 AWS Secret Access Key (password). 36216 Leave blank for anonymous access or runtime credentials. 36217 Enter a value. Press Enter to leave empty. 36218 secret_access_key> secretaccesskey 36219 Option endpoint. 36220 Endpoint for China Mobile Ecloud Elastic Object Storage (EOS) API. 36221 Choose a number from below, or type in your own value. 36222 Press Enter to leave empty. 36223 / The default endpoint - a good choice if you are unsure. 36224 1 | East China (Suzhou) 36225 \[rs] (eos-wuxi-1.cmecloud.cn) 36226 2 / East China (Jinan) 36227 \[rs] (eos-jinan-1.cmecloud.cn) 36228 3 / East China (Hangzhou) 36229 \[rs] (eos-ningbo-1.cmecloud.cn) 36230 4 / East China (Shanghai-1) 36231 \[rs] (eos-shanghai-1.cmecloud.cn) 36232 5 / Central China (Zhengzhou) 36233 \[rs] (eos-zhengzhou-1.cmecloud.cn) 36234 6 / Central China (Changsha-1) 36235 \[rs] (eos-hunan-1.cmecloud.cn) 36236 7 / Central China (Changsha-2) 36237 \[rs] (eos-zhuzhou-1.cmecloud.cn) 36238 8 / South China (Guangzhou-2) 36239 \[rs] (eos-guangzhou-1.cmecloud.cn) 36240 9 / South China (Guangzhou-3) 36241 \[rs] (eos-dongguan-1.cmecloud.cn) 36242 10 / North China (Beijing-1) 36243 \[rs] (eos-beijing-1.cmecloud.cn) 36244 11 / North China (Beijing-2) 36245 \[rs] (eos-beijing-2.cmecloud.cn) 36246 12 / North China (Beijing-3) 36247 \[rs] (eos-beijing-4.cmecloud.cn) 36248 13 / North China (Huhehaote) 36249 \[rs] (eos-huhehaote-1.cmecloud.cn) 36250 14 / Southwest China (Chengdu) 36251 \[rs] (eos-chengdu-1.cmecloud.cn) 36252 15 / Southwest China (Chongqing) 36253 \[rs] (eos-chongqing-1.cmecloud.cn) 36254 16 / Southwest China (Guiyang) 36255 \[rs] (eos-guiyang-1.cmecloud.cn) 36256 17 / Nouthwest China (Xian) 36257 \[rs] (eos-xian-1.cmecloud.cn) 36258 18 / Yunnan China (Kunming) 36259 \[rs] (eos-yunnan.cmecloud.cn) 36260 19 / Yunnan China (Kunming-2) 36261 \[rs] (eos-yunnan-2.cmecloud.cn) 36262 20 / Tianjin China (Tianjin) 36263 \[rs] (eos-tianjin-1.cmecloud.cn) 36264 21 / Jilin China (Changchun) 36265 \[rs] (eos-jilin-1.cmecloud.cn) 36266 22 / Hubei China (Xiangyan) 36267 \[rs] (eos-hubei-1.cmecloud.cn) 36268 23 / Jiangxi China (Nanchang) 36269 \[rs] (eos-jiangxi-1.cmecloud.cn) 36270 24 / Gansu China (Lanzhou) 36271 \[rs] (eos-gansu-1.cmecloud.cn) 36272 25 / Shanxi China (Taiyuan) 36273 \[rs] (eos-shanxi-1.cmecloud.cn) 36274 26 / Liaoning China (Shenyang) 36275 \[rs] (eos-liaoning-1.cmecloud.cn) 36276 27 / Hebei China (Shijiazhuang) 36277 \[rs] (eos-hebei-1.cmecloud.cn) 36278 28 / Fujian China (Xiamen) 36279 \[rs] (eos-fujian-1.cmecloud.cn) 36280 29 / Guangxi China (Nanning) 36281 \[rs] (eos-guangxi-1.cmecloud.cn) 36282 30 / Anhui China (Huainan) 36283 \[rs] (eos-anhui-1.cmecloud.cn) 36284 endpoint> 1 36285 Option location_constraint. 36286 Location constraint - must match endpoint. 36287 Used when creating buckets only. 36288 Choose a number from below, or type in your own value. 36289 Press Enter to leave empty. 36290 1 / East China (Suzhou) 36291 \[rs] (wuxi1) 36292 2 / East China (Jinan) 36293 \[rs] (jinan1) 36294 3 / East China (Hangzhou) 36295 \[rs] (ningbo1) 36296 4 / East China (Shanghai-1) 36297 \[rs] (shanghai1) 36298 5 / Central China (Zhengzhou) 36299 \[rs] (zhengzhou1) 36300 6 / Central China (Changsha-1) 36301 \[rs] (hunan1) 36302 7 / Central China (Changsha-2) 36303 \[rs] (zhuzhou1) 36304 8 / South China (Guangzhou-2) 36305 \[rs] (guangzhou1) 36306 9 / South China (Guangzhou-3) 36307 \[rs] (dongguan1) 36308 10 / North China (Beijing-1) 36309 \[rs] (beijing1) 36310 11 / North China (Beijing-2) 36311 \[rs] (beijing2) 36312 12 / North China (Beijing-3) 36313 \[rs] (beijing4) 36314 13 / North China (Huhehaote) 36315 \[rs] (huhehaote1) 36316 14 / Southwest China (Chengdu) 36317 \[rs] (chengdu1) 36318 15 / Southwest China (Chongqing) 36319 \[rs] (chongqing1) 36320 16 / Southwest China (Guiyang) 36321 \[rs] (guiyang1) 36322 17 / Nouthwest China (Xian) 36323 \[rs] (xian1) 36324 18 / Yunnan China (Kunming) 36325 \[rs] (yunnan) 36326 19 / Yunnan China (Kunming-2) 36327 \[rs] (yunnan2) 36328 20 / Tianjin China (Tianjin) 36329 \[rs] (tianjin1) 36330 21 / Jilin China (Changchun) 36331 \[rs] (jilin1) 36332 22 / Hubei China (Xiangyan) 36333 \[rs] (hubei1) 36334 23 / Jiangxi China (Nanchang) 36335 \[rs] (jiangxi1) 36336 24 / Gansu China (Lanzhou) 36337 \[rs] (gansu1) 36338 25 / Shanxi China (Taiyuan) 36339 \[rs] (shanxi1) 36340 26 / Liaoning China (Shenyang) 36341 \[rs] (liaoning1) 36342 27 / Hebei China (Shijiazhuang) 36343 \[rs] (hebei1) 36344 28 / Fujian China (Xiamen) 36345 \[rs] (fujian1) 36346 29 / Guangxi China (Nanning) 36347 \[rs] (guangxi1) 36348 30 / Anhui China (Huainan) 36349 \[rs] (anhui1) 36350 location_constraint> 1 36351 Option acl. 36352 Canned ACL used when creating buckets and storing or copying objects. 36353 This ACL is used for creating objects and if bucket_acl isn\[aq]t set, for creating buckets too. 36354 For more info visit https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl 36355 Note that this ACL is applied when server-side copying objects as S3 36356 doesn\[aq]t copy the ACL from the source but rather writes a fresh one. 36357 Choose a number from below, or type in your own value. 36358 Press Enter to leave empty. 36359 / Owner gets FULL_CONTROL. 36360 1 | No one else has access rights (default). 36361 \[rs] (private) 36362 / Owner gets FULL_CONTROL. 36363 2 | The AllUsers group gets READ access. 36364 \[rs] (public-read) 36365 / Owner gets FULL_CONTROL. 36366 3 | The AllUsers group gets READ and WRITE access. 36367 | Granting this on a bucket is generally not recommended. 36368 \[rs] (public-read-write) 36369 / Owner gets FULL_CONTROL. 36370 4 | The AuthenticatedUsers group gets READ access. 36371 \[rs] (authenticated-read) 36372 / Object owner gets FULL_CONTROL. 36373 acl> private 36374 Option server_side_encryption. 36375 The server-side encryption algorithm used when storing this object in S3. 36376 Choose a number from below, or type in your own value. 36377 Press Enter to leave empty. 36378 1 / None 36379 \[rs] () 36380 2 / AES256 36381 \[rs] (AES256) 36382 server_side_encryption> 36383 Option storage_class. 36384 The storage class to use when storing new objects in ChinaMobile. 36385 Choose a number from below, or type in your own value. 36386 Press Enter to leave empty. 36387 1 / Default 36388 \[rs] () 36389 2 / Standard storage class 36390 \[rs] (STANDARD) 36391 3 / Archive storage mode 36392 \[rs] (GLACIER) 36393 4 / Infrequent access storage mode 36394 \[rs] (STANDARD_IA) 36395 storage_class> 36396 Edit advanced config? 36397 y) Yes 36398 n) No (default) 36399 y/n> n 36400 -------------------- 36401 [ChinaMobile] 36402 type = s3 36403 provider = ChinaMobile 36404 access_key_id = accesskeyid 36405 secret_access_key = secretaccesskey 36406 endpoint = eos-wuxi-1.cmecloud.cn 36407 location_constraint = wuxi1 36408 acl = private 36409 -------------------- 36410 y) Yes this is OK (default) 36411 e) Edit this remote 36412 d) Delete this remote 36413 y/e/d> y 36414 \f[R] 36415 .fi 36416 .SS Leviia Cloud Object Storage 36417 .PP 36418 Leviia Object Storage (https://www.leviia.com/object-storage/), backup 36419 and secure your data in a 100% French cloud, independent of GAFAM.. 36420 .PP 36421 To configure access to Leviia, follow the steps below: 36422 .IP "1." 3 36423 Run \f[C]rclone config\f[R] and select \f[C]n\f[R] for a new remote. 36424 .IP 36425 .nf 36426 \f[C] 36427 rclone config 36428 No remotes found, make a new one? 36429 n) New remote 36430 s) Set configuration password 36431 q) Quit config 36432 n/s/q> n 36433 \f[R] 36434 .fi 36435 .IP "2." 3 36436 Give the name of the configuration. 36437 For example, name it \[aq]leviia\[aq]. 36438 .IP 36439 .nf 36440 \f[C] 36441 name> leviia 36442 \f[R] 36443 .fi 36444 .IP "3." 3 36445 Select \f[C]s3\f[R] storage. 36446 .IP 36447 .nf 36448 \f[C] 36449 Choose a number from below, or type in your own value 36450 [snip] 36451 XX / Amazon S3 Compliant Storage Providers including AWS, ... 36452 \[rs] (s3) 36453 [snip] 36454 Storage> s3 36455 \f[R] 36456 .fi 36457 .IP "4." 3 36458 Select \f[C]Leviia\f[R] provider. 36459 .IP 36460 .nf 36461 \f[C] 36462 Choose a number from below, or type in your own value 36463 1 / Amazon Web Services (AWS) S3 36464 \[rs] \[dq]AWS\[dq] 36465 [snip] 36466 15 / Leviia Object Storage 36467 \[rs] (Leviia) 36468 [snip] 36469 provider> Leviia 36470 \f[R] 36471 .fi 36472 .IP "5." 3 36473 Enter your SecretId and SecretKey of Leviia. 36474 .IP 36475 .nf 36476 \f[C] 36477 Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars). 36478 Only applies if access_key_id and secret_access_key is blank. 36479 Enter a boolean value (true or false). Press Enter for the default (\[dq]false\[dq]). 36480 Choose a number from below, or type in your own value 36481 1 / Enter AWS credentials in the next step 36482 \[rs] \[dq]false\[dq] 36483 2 / Get AWS credentials from the environment (env vars or IAM) 36484 \[rs] \[dq]true\[dq] 36485 env_auth> 1 36486 AWS Access Key ID. 36487 Leave blank for anonymous access or runtime credentials. 36488 Enter a string value. Press Enter for the default (\[dq]\[dq]). 36489 access_key_id> ZnIx.xxxxxxxxxxxxxxx 36490 AWS Secret Access Key (password) 36491 Leave blank for anonymous access or runtime credentials. 36492 Enter a string value. Press Enter for the default (\[dq]\[dq]). 36493 secret_access_key> xxxxxxxxxxx 36494 \f[R] 36495 .fi 36496 .IP "6." 3 36497 Select endpoint for Leviia. 36498 .IP 36499 .nf 36500 \f[C] 36501 / The default endpoint 36502 1 | Leviia. 36503 \[rs] (s3.leviia.com) 36504 [snip] 36505 endpoint> 1 36506 \f[R] 36507 .fi 36508 .IP "7." 3 36509 Choose acl. 36510 .IP 36511 .nf 36512 \f[C] 36513 Note that this ACL is applied when server-side copying objects as S3 36514 doesn\[aq]t copy the ACL from the source but rather writes a fresh one. 36515 Enter a string value. Press Enter for the default (\[dq]\[dq]). 36516 Choose a number from below, or type in your own value 36517 / Owner gets FULL_CONTROL. 36518 1 | No one else has access rights (default). 36519 \[rs] (private) 36520 / Owner gets FULL_CONTROL. 36521 2 | The AllUsers group gets READ access. 36522 \[rs] (public-read) 36523 [snip] 36524 acl> 1 36525 Edit advanced config? (y/n) 36526 y) Yes 36527 n) No (default) 36528 y/n> n 36529 Remote config 36530 -------------------- 36531 [leviia] 36532 - type: s3 36533 - provider: Leviia 36534 - access_key_id: ZnIx.xxxxxxx 36535 - secret_access_key: xxxxxxxx 36536 - endpoint: s3.leviia.com 36537 - acl: private 36538 -------------------- 36539 y) Yes this is OK (default) 36540 e) Edit this remote 36541 d) Delete this remote 36542 y/e/d> y 36543 Current remotes: 36544 36545 Name Type 36546 ==== ==== 36547 leviia s3 36548 \f[R] 36549 .fi 36550 .SS Liara 36551 .PP 36552 Here is an example of making a Liara Object 36553 Storage (https://liara.ir/landing/object-storage) configuration. 36554 First run: 36555 .IP 36556 .nf 36557 \f[C] 36558 rclone config 36559 \f[R] 36560 .fi 36561 .PP 36562 This will guide you through an interactive setup process. 36563 .IP 36564 .nf 36565 \f[C] 36566 No remotes found, make a new one? 36567 n) New remote 36568 s) Set configuration password 36569 n/s> n 36570 name> Liara 36571 Type of storage to configure. 36572 Choose a number from below, or type in your own value 36573 [snip] 36574 XX / Amazon S3 (also Dreamhost, Ceph, ChinaMobile, ArvanCloud, Liara, Minio) 36575 \[rs] \[dq]s3\[dq] 36576 [snip] 36577 Storage> s3 36578 Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars). Only applies if access_key_id and secret_access_key is blank. 36579 Choose a number from below, or type in your own value 36580 1 / Enter AWS credentials in the next step 36581 \[rs] \[dq]false\[dq] 36582 2 / Get AWS credentials from the environment (env vars or IAM) 36583 \[rs] \[dq]true\[dq] 36584 env_auth> 1 36585 AWS Access Key ID - leave blank for anonymous access or runtime credentials. 36586 access_key_id> YOURACCESSKEY 36587 AWS Secret Access Key (password) - leave blank for anonymous access or runtime credentials. 36588 secret_access_key> YOURSECRETACCESSKEY 36589 Region to connect to. 36590 Choose a number from below, or type in your own value 36591 / The default endpoint 36592 1 | US Region, Northern Virginia, or Pacific Northwest. 36593 | Leave location constraint empty. 36594 \[rs] \[dq]us-east-1\[dq] 36595 [snip] 36596 region> 36597 Endpoint for S3 API. 36598 Leave blank if using Liara to use the default endpoint for the region. 36599 Specify if using an S3 clone such as Ceph. 36600 endpoint> storage.iran.liara.space 36601 Canned ACL used when creating buckets and/or storing objects in S3. 36602 For more info visit https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl 36603 Choose a number from below, or type in your own value 36604 1 / Owner gets FULL_CONTROL. No one else has access rights (default). 36605 \[rs] \[dq]private\[dq] 36606 [snip] 36607 acl> 36608 The server-side encryption algorithm used when storing this object in S3. 36609 Choose a number from below, or type in your own value 36610 1 / None 36611 \[rs] \[dq]\[dq] 36612 2 / AES256 36613 \[rs] \[dq]AES256\[dq] 36614 server_side_encryption> 36615 The storage class to use when storing objects in S3. 36616 Choose a number from below, or type in your own value 36617 1 / Default 36618 \[rs] \[dq]\[dq] 36619 2 / Standard storage class 36620 \[rs] \[dq]STANDARD\[dq] 36621 storage_class> 36622 Remote config 36623 -------------------- 36624 [Liara] 36625 env_auth = false 36626 access_key_id = YOURACCESSKEY 36627 secret_access_key = YOURSECRETACCESSKEY 36628 endpoint = storage.iran.liara.space 36629 location_constraint = 36630 acl = 36631 server_side_encryption = 36632 storage_class = 36633 -------------------- 36634 y) Yes this is OK 36635 e) Edit this remote 36636 d) Delete this remote 36637 y/e/d> y 36638 \f[R] 36639 .fi 36640 .PP 36641 This will leave the config file looking like this. 36642 .IP 36643 .nf 36644 \f[C] 36645 [Liara] 36646 type = s3 36647 provider = Liara 36648 env_auth = false 36649 access_key_id = YOURACCESSKEY 36650 secret_access_key = YOURSECRETACCESSKEY 36651 region = 36652 endpoint = storage.iran.liara.space 36653 location_constraint = 36654 acl = 36655 server_side_encryption = 36656 storage_class = 36657 \f[R] 36658 .fi 36659 .SS Linode 36660 .PP 36661 Here is an example of making a Linode Object 36662 Storage (https://www.linode.com/products/object-storage/) configuration. 36663 First run: 36664 .IP 36665 .nf 36666 \f[C] 36667 rclone config 36668 \f[R] 36669 .fi 36670 .PP 36671 This will guide you through an interactive setup process. 36672 .IP 36673 .nf 36674 \f[C] 36675 No remotes found, make a new one? 36676 n) New remote 36677 s) Set configuration password 36678 q) Quit config 36679 n/s/q> n 36680 36681 Enter name for new remote. 36682 name> linode 36683 36684 Option Storage. 36685 Type of storage to configure. 36686 Choose a number from below, or type in your own value. 36687 [snip] 36688 XX / Amazon S3 Compliant Storage Providers including AWS, ...Linode, ...and others 36689 \[rs] (s3) 36690 [snip] 36691 Storage> s3 36692 36693 Option provider. 36694 Choose your S3 provider. 36695 Choose a number from below, or type in your own value. 36696 Press Enter to leave empty. 36697 [snip] 36698 XX / Linode Object Storage 36699 \[rs] (Linode) 36700 [snip] 36701 provider> Linode 36702 36703 Option env_auth. 36704 Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars). 36705 Only applies if access_key_id and secret_access_key is blank. 36706 Choose a number from below, or type in your own boolean value (true or false). 36707 Press Enter for the default (false). 36708 1 / Enter AWS credentials in the next step. 36709 \[rs] (false) 36710 2 / Get AWS credentials from the environment (env vars or IAM). 36711 \[rs] (true) 36712 env_auth> 36713 36714 Option access_key_id. 36715 AWS Access Key ID. 36716 Leave blank for anonymous access or runtime credentials. 36717 Enter a value. Press Enter to leave empty. 36718 access_key_id> ACCESS_KEY 36719 36720 Option secret_access_key. 36721 AWS Secret Access Key (password). 36722 Leave blank for anonymous access or runtime credentials. 36723 Enter a value. Press Enter to leave empty. 36724 secret_access_key> SECRET_ACCESS_KEY 36725 36726 Option endpoint. 36727 Endpoint for Linode Object Storage API. 36728 Choose a number from below, or type in your own value. 36729 Press Enter to leave empty. 36730 1 / Atlanta, GA (USA), us-southeast-1 36731 \[rs] (us-southeast-1.linodeobjects.com) 36732 2 / Chicago, IL (USA), us-ord-1 36733 \[rs] (us-ord-1.linodeobjects.com) 36734 3 / Frankfurt (Germany), eu-central-1 36735 \[rs] (eu-central-1.linodeobjects.com) 36736 4 / Milan (Italy), it-mil-1 36737 \[rs] (it-mil-1.linodeobjects.com) 36738 5 / Newark, NJ (USA), us-east-1 36739 \[rs] (us-east-1.linodeobjects.com) 36740 6 / Paris (France), fr-par-1 36741 \[rs] (fr-par-1.linodeobjects.com) 36742 7 / Seattle, WA (USA), us-sea-1 36743 \[rs] (us-sea-1.linodeobjects.com) 36744 8 / Singapore ap-south-1 36745 \[rs] (ap-south-1.linodeobjects.com) 36746 9 / Stockholm (Sweden), se-sto-1 36747 \[rs] (se-sto-1.linodeobjects.com) 36748 10 / Washington, DC, (USA), us-iad-1 36749 \[rs] (us-iad-1.linodeobjects.com) 36750 endpoint> 3 36751 36752 Option acl. 36753 Canned ACL used when creating buckets and storing or copying objects. 36754 This ACL is used for creating objects and if bucket_acl isn\[aq]t set, for creating buckets too. 36755 For more info visit https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl 36756 Note that this ACL is applied when server-side copying objects as S3 36757 doesn\[aq]t copy the ACL from the source but rather writes a fresh one. 36758 If the acl is an empty string then no X-Amz-Acl: header is added and 36759 the default (private) will be used. 36760 Choose a number from below, or type in your own value. 36761 Press Enter to leave empty. 36762 / Owner gets FULL_CONTROL. 36763 1 | No one else has access rights (default). 36764 \[rs] (private) 36765 [snip] 36766 acl> 36767 36768 Edit advanced config? 36769 y) Yes 36770 n) No (default) 36771 y/n> n 36772 36773 Configuration complete. 36774 Options: 36775 - type: s3 36776 - provider: Linode 36777 - access_key_id: ACCESS_KEY 36778 - secret_access_key: SECRET_ACCESS_KEY 36779 - endpoint: eu-central-1.linodeobjects.com 36780 Keep this \[dq]linode\[dq] remote? 36781 y) Yes this is OK (default) 36782 e) Edit this remote 36783 d) Delete this remote 36784 y/e/d> y 36785 \f[R] 36786 .fi 36787 .PP 36788 This will leave the config file looking like this. 36789 .IP 36790 .nf 36791 \f[C] 36792 [linode] 36793 type = s3 36794 provider = Linode 36795 access_key_id = ACCESS_KEY 36796 secret_access_key = SECRET_ACCESS_KEY 36797 endpoint = eu-central-1.linodeobjects.com 36798 \f[R] 36799 .fi 36800 .SS ArvanCloud 36801 .PP 36802 ArvanCloud (https://www.arvancloud.com/en/products/cloud-storage) 36803 ArvanCloud Object Storage goes beyond the limited traditional file 36804 storage. 36805 It gives you access to backup and archived files and allows sharing. 36806 Files like profile image in the app, images sent by users or scanned 36807 documents can be stored securely and easily in our Object Storage 36808 service. 36809 .PP 36810 ArvanCloud provides an S3 interface which can be configured for use with 36811 rclone like this. 36812 .IP 36813 .nf 36814 \f[C] 36815 No remotes found, make a new one? 36816 n) New remote 36817 s) Set configuration password 36818 n/s> n 36819 name> ArvanCloud 36820 Type of storage to configure. 36821 Choose a number from below, or type in your own value 36822 [snip] 36823 XX / Amazon S3 (also Dreamhost, Ceph, ChinaMobile, ArvanCloud, Liara, Minio) 36824 \[rs] \[dq]s3\[dq] 36825 [snip] 36826 Storage> s3 36827 Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars). Only applies if access_key_id and secret_access_key is blank. 36828 Choose a number from below, or type in your own value 36829 1 / Enter AWS credentials in the next step 36830 \[rs] \[dq]false\[dq] 36831 2 / Get AWS credentials from the environment (env vars or IAM) 36832 \[rs] \[dq]true\[dq] 36833 env_auth> 1 36834 AWS Access Key ID - leave blank for anonymous access or runtime credentials. 36835 access_key_id> YOURACCESSKEY 36836 AWS Secret Access Key (password) - leave blank for anonymous access or runtime credentials. 36837 secret_access_key> YOURSECRETACCESSKEY 36838 Region to connect to. 36839 Choose a number from below, or type in your own value 36840 / The default endpoint - a good choice if you are unsure. 36841 1 | US Region, Northern Virginia, or Pacific Northwest. 36842 | Leave location constraint empty. 36843 \[rs] \[dq]us-east-1\[dq] 36844 [snip] 36845 region> 36846 Endpoint for S3 API. 36847 Leave blank if using ArvanCloud to use the default endpoint for the region. 36848 Specify if using an S3 clone such as Ceph. 36849 endpoint> s3.arvanstorage.com 36850 Location constraint - must be set to match the Region. Used when creating buckets only. 36851 Choose a number from below, or type in your own value 36852 1 / Empty for Iran-Tehran Region. 36853 \[rs] \[dq]\[dq] 36854 [snip] 36855 location_constraint> 36856 Canned ACL used when creating buckets and/or storing objects in S3. 36857 For more info visit https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl 36858 Choose a number from below, or type in your own value 36859 1 / Owner gets FULL_CONTROL. No one else has access rights (default). 36860 \[rs] \[dq]private\[dq] 36861 [snip] 36862 acl> 36863 The server-side encryption algorithm used when storing this object in S3. 36864 Choose a number from below, or type in your own value 36865 1 / None 36866 \[rs] \[dq]\[dq] 36867 2 / AES256 36868 \[rs] \[dq]AES256\[dq] 36869 server_side_encryption> 36870 The storage class to use when storing objects in S3. 36871 Choose a number from below, or type in your own value 36872 1 / Default 36873 \[rs] \[dq]\[dq] 36874 2 / Standard storage class 36875 \[rs] \[dq]STANDARD\[dq] 36876 storage_class> 36877 Remote config 36878 -------------------- 36879 [ArvanCloud] 36880 env_auth = false 36881 access_key_id = YOURACCESSKEY 36882 secret_access_key = YOURSECRETACCESSKEY 36883 region = ir-thr-at1 36884 endpoint = s3.arvanstorage.com 36885 location_constraint = 36886 acl = 36887 server_side_encryption = 36888 storage_class = 36889 -------------------- 36890 y) Yes this is OK 36891 e) Edit this remote 36892 d) Delete this remote 36893 y/e/d> y 36894 \f[R] 36895 .fi 36896 .PP 36897 This will leave the config file looking like this. 36898 .IP 36899 .nf 36900 \f[C] 36901 [ArvanCloud] 36902 type = s3 36903 provider = ArvanCloud 36904 env_auth = false 36905 access_key_id = YOURACCESSKEY 36906 secret_access_key = YOURSECRETACCESSKEY 36907 region = 36908 endpoint = s3.arvanstorage.com 36909 location_constraint = 36910 acl = 36911 server_side_encryption = 36912 storage_class = 36913 \f[R] 36914 .fi 36915 .SS Tencent COS 36916 .PP 36917 Tencent Cloud Object Storage 36918 (COS) (https://intl.cloud.tencent.com/product/cos) is a distributed 36919 storage service offered by Tencent Cloud for unstructured data. 36920 It is secure, stable, massive, convenient, low-delay and low-cost. 36921 .PP 36922 To configure access to Tencent COS, follow the steps below: 36923 .IP "1." 3 36924 Run \f[C]rclone config\f[R] and select \f[C]n\f[R] for a new remote. 36925 .IP 36926 .nf 36927 \f[C] 36928 rclone config 36929 No remotes found, make a new one? 36930 n) New remote 36931 s) Set configuration password 36932 q) Quit config 36933 n/s/q> n 36934 \f[R] 36935 .fi 36936 .IP "2." 3 36937 Give the name of the configuration. 36938 For example, name it \[aq]cos\[aq]. 36939 .IP 36940 .nf 36941 \f[C] 36942 name> cos 36943 \f[R] 36944 .fi 36945 .IP "3." 3 36946 Select \f[C]s3\f[R] storage. 36947 .IP 36948 .nf 36949 \f[C] 36950 Choose a number from below, or type in your own value 36951 [snip] 36952 XX / Amazon S3 Compliant Storage Providers including AWS, ... 36953 \[rs] \[dq]s3\[dq] 36954 [snip] 36955 Storage> s3 36956 \f[R] 36957 .fi 36958 .IP "4." 3 36959 Select \f[C]TencentCOS\f[R] provider. 36960 .IP 36961 .nf 36962 \f[C] 36963 Choose a number from below, or type in your own value 36964 1 / Amazon Web Services (AWS) S3 36965 \[rs] \[dq]AWS\[dq] 36966 [snip] 36967 11 / Tencent Cloud Object Storage (COS) 36968 \[rs] \[dq]TencentCOS\[dq] 36969 [snip] 36970 provider> TencentCOS 36971 \f[R] 36972 .fi 36973 .IP "5." 3 36974 Enter your SecretId and SecretKey of Tencent Cloud. 36975 .IP 36976 .nf 36977 \f[C] 36978 Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars). 36979 Only applies if access_key_id and secret_access_key is blank. 36980 Enter a boolean value (true or false). Press Enter for the default (\[dq]false\[dq]). 36981 Choose a number from below, or type in your own value 36982 1 / Enter AWS credentials in the next step 36983 \[rs] \[dq]false\[dq] 36984 2 / Get AWS credentials from the environment (env vars or IAM) 36985 \[rs] \[dq]true\[dq] 36986 env_auth> 1 36987 AWS Access Key ID. 36988 Leave blank for anonymous access or runtime credentials. 36989 Enter a string value. Press Enter for the default (\[dq]\[dq]). 36990 access_key_id> AKIDxxxxxxxxxx 36991 AWS Secret Access Key (password) 36992 Leave blank for anonymous access or runtime credentials. 36993 Enter a string value. Press Enter for the default (\[dq]\[dq]). 36994 secret_access_key> xxxxxxxxxxx 36995 \f[R] 36996 .fi 36997 .IP "6." 3 36998 Select endpoint for Tencent COS. 36999 This is the standard endpoint for different region. 37000 .IP 37001 .nf 37002 \f[C] 37003 1 / Beijing Region. 37004 \[rs] \[dq]cos.ap-beijing.myqcloud.com\[dq] 37005 2 / Nanjing Region. 37006 \[rs] \[dq]cos.ap-nanjing.myqcloud.com\[dq] 37007 3 / Shanghai Region. 37008 \[rs] \[dq]cos.ap-shanghai.myqcloud.com\[dq] 37009 4 / Guangzhou Region. 37010 \[rs] \[dq]cos.ap-guangzhou.myqcloud.com\[dq] 37011 [snip] 37012 endpoint> 4 37013 \f[R] 37014 .fi 37015 .IP "7." 3 37016 Choose acl and storage class. 37017 .IP 37018 .nf 37019 \f[C] 37020 Note that this ACL is applied when server-side copying objects as S3 37021 doesn\[aq]t copy the ACL from the source but rather writes a fresh one. 37022 Enter a string value. Press Enter for the default (\[dq]\[dq]). 37023 Choose a number from below, or type in your own value 37024 1 / Owner gets Full_CONTROL. No one else has access rights (default). 37025 \[rs] \[dq]default\[dq] 37026 [snip] 37027 acl> 1 37028 The storage class to use when storing new objects in Tencent COS. 37029 Enter a string value. Press Enter for the default (\[dq]\[dq]). 37030 Choose a number from below, or type in your own value 37031 1 / Default 37032 \[rs] \[dq]\[dq] 37033 [snip] 37034 storage_class> 1 37035 Edit advanced config? (y/n) 37036 y) Yes 37037 n) No (default) 37038 y/n> n 37039 Remote config 37040 -------------------- 37041 [cos] 37042 type = s3 37043 provider = TencentCOS 37044 env_auth = false 37045 access_key_id = xxx 37046 secret_access_key = xxx 37047 endpoint = cos.ap-guangzhou.myqcloud.com 37048 acl = default 37049 -------------------- 37050 y) Yes this is OK (default) 37051 e) Edit this remote 37052 d) Delete this remote 37053 y/e/d> y 37054 Current remotes: 37055 37056 Name Type 37057 ==== ==== 37058 cos s3 37059 \f[R] 37060 .fi 37061 .SS Netease NOS 37062 .PP 37063 For Netease NOS configure as per the configurator 37064 \f[C]rclone config\f[R] setting the provider \f[C]Netease\f[R]. 37065 This will automatically set \f[C]force_path_style = false\f[R] which is 37066 necessary for it to run properly. 37067 .SS Petabox 37068 .PP 37069 Here is an example of making a Petabox (https://petabox.io/) 37070 configuration. 37071 First run: 37072 .IP 37073 .nf 37074 \f[C] 37075 rclone config 37076 \f[R] 37077 .fi 37078 .PP 37079 This will guide you through an interactive setup process. 37080 .IP 37081 .nf 37082 \f[C] 37083 No remotes found, make a new one? 37084 n) New remote 37085 s) Set configuration password 37086 n/s> n 37087 37088 Enter name for new remote. 37089 name> My Petabox Storage 37090 37091 Option Storage. 37092 Type of storage to configure. 37093 Choose a number from below, or type in your own value. 37094 [snip] 37095 XX / Amazon S3 Compliant Storage Providers including AWS, ... 37096 \[rs] \[dq]s3\[dq] 37097 [snip] 37098 Storage> s3 37099 37100 Option provider. 37101 Choose your S3 provider. 37102 Choose a number from below, or type in your own value. 37103 Press Enter to leave empty. 37104 [snip] 37105 XX / Petabox Object Storage 37106 \[rs] (Petabox) 37107 [snip] 37108 provider> Petabox 37109 37110 Option env_auth. 37111 Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars). 37112 Only applies if access_key_id and secret_access_key is blank. 37113 Choose a number from below, or type in your own boolean value (true or false). 37114 Press Enter for the default (false). 37115 1 / Enter AWS credentials in the next step. 37116 \[rs] (false) 37117 2 / Get AWS credentials from the environment (env vars or IAM). 37118 \[rs] (true) 37119 env_auth> 1 37120 37121 Option access_key_id. 37122 AWS Access Key ID. 37123 Leave blank for anonymous access or runtime credentials. 37124 Enter a value. Press Enter to leave empty. 37125 access_key_id> YOUR_ACCESS_KEY_ID 37126 37127 Option secret_access_key. 37128 AWS Secret Access Key (password). 37129 Leave blank for anonymous access or runtime credentials. 37130 Enter a value. Press Enter to leave empty. 37131 secret_access_key> YOUR_SECRET_ACCESS_KEY 37132 37133 Option region. 37134 Region where your bucket will be created and your data stored. 37135 Choose a number from below, or type in your own value. 37136 Press Enter to leave empty. 37137 1 / US East (N. Virginia) 37138 \[rs] (us-east-1) 37139 2 / Europe (Frankfurt) 37140 \[rs] (eu-central-1) 37141 3 / Asia Pacific (Singapore) 37142 \[rs] (ap-southeast-1) 37143 4 / Middle East (Bahrain) 37144 \[rs] (me-south-1) 37145 5 / South America (S\[~a]o Paulo) 37146 \[rs] (sa-east-1) 37147 region> 1 37148 37149 Option endpoint. 37150 Endpoint for Petabox S3 Object Storage. 37151 Specify the endpoint from the same region. 37152 Choose a number from below, or type in your own value. 37153 1 / US East (N. Virginia) 37154 \[rs] (s3.petabox.io) 37155 2 / US East (N. Virginia) 37156 \[rs] (s3.us-east-1.petabox.io) 37157 3 / Europe (Frankfurt) 37158 \[rs] (s3.eu-central-1.petabox.io) 37159 4 / Asia Pacific (Singapore) 37160 \[rs] (s3.ap-southeast-1.petabox.io) 37161 5 / Middle East (Bahrain) 37162 \[rs] (s3.me-south-1.petabox.io) 37163 6 / South America (S\[~a]o Paulo) 37164 \[rs] (s3.sa-east-1.petabox.io) 37165 endpoint> 1 37166 37167 Option acl. 37168 Canned ACL used when creating buckets and storing or copying objects. 37169 This ACL is used for creating objects and if bucket_acl isn\[aq]t set, for creating buckets too. 37170 For more info visit https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl 37171 Note that this ACL is applied when server-side copying objects as S3 37172 doesn\[aq]t copy the ACL from the source but rather writes a fresh one. 37173 If the acl is an empty string then no X-Amz-Acl: header is added and 37174 the default (private) will be used. 37175 Choose a number from below, or type in your own value. 37176 Press Enter to leave empty. 37177 / Owner gets FULL_CONTROL. 37178 1 | No one else has access rights (default). 37179 \[rs] (private) 37180 / Owner gets FULL_CONTROL. 37181 2 | The AllUsers group gets READ access. 37182 \[rs] (public-read) 37183 / Owner gets FULL_CONTROL. 37184 3 | The AllUsers group gets READ and WRITE access. 37185 | Granting this on a bucket is generally not recommended. 37186 \[rs] (public-read-write) 37187 / Owner gets FULL_CONTROL. 37188 4 | The AuthenticatedUsers group gets READ access. 37189 \[rs] (authenticated-read) 37190 / Object owner gets FULL_CONTROL. 37191 5 | Bucket owner gets READ access. 37192 | If you specify this canned ACL when creating a bucket, Amazon S3 ignores it. 37193 \[rs] (bucket-owner-read) 37194 / Both the object owner and the bucket owner get FULL_CONTROL over the object. 37195 6 | If you specify this canned ACL when creating a bucket, Amazon S3 ignores it. 37196 \[rs] (bucket-owner-full-control) 37197 acl> 1 37198 37199 Edit advanced config? 37200 y) Yes 37201 n) No (default) 37202 y/n> No 37203 37204 Configuration complete. 37205 Options: 37206 - type: s3 37207 - provider: Petabox 37208 - access_key_id: YOUR_ACCESS_KEY_ID 37209 - secret_access_key: YOUR_SECRET_ACCESS_KEY 37210 - region: us-east-1 37211 - endpoint: s3.petabox.io 37212 Keep this \[dq]My Petabox Storage\[dq] remote? 37213 y) Yes this is OK (default) 37214 e) Edit this remote 37215 d) Delete this remote 37216 y/e/d> y 37217 \f[R] 37218 .fi 37219 .PP 37220 This will leave the config file looking like this. 37221 .IP 37222 .nf 37223 \f[C] 37224 [My Petabox Storage] 37225 type = s3 37226 provider = Petabox 37227 access_key_id = YOUR_ACCESS_KEY_ID 37228 secret_access_key = YOUR_SECRET_ACCESS_KEY 37229 region = us-east-1 37230 endpoint = s3.petabox.io 37231 \f[R] 37232 .fi 37233 .SS Storj 37234 .PP 37235 Storj is a decentralized cloud storage which can be used through its 37236 native protocol or an S3 compatible gateway. 37237 .PP 37238 The S3 compatible gateway is configured using \f[C]rclone config\f[R] 37239 with a type of \f[C]s3\f[R] and with a provider name of \f[C]Storj\f[R]. 37240 Here is an example run of the configurator. 37241 .IP 37242 .nf 37243 \f[C] 37244 Type of storage to configure. 37245 Storage> s3 37246 Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars). 37247 Only applies if access_key_id and secret_access_key is blank. 37248 Choose a number from below, or type in your own boolean value (true or false). 37249 Press Enter for the default (false). 37250 1 / Enter AWS credentials in the next step. 37251 \[rs] (false) 37252 2 / Get AWS credentials from the environment (env vars or IAM). 37253 \[rs] (true) 37254 env_auth> 1 37255 Option access_key_id. 37256 AWS Access Key ID. 37257 Leave blank for anonymous access or runtime credentials. 37258 Enter a value. Press Enter to leave empty. 37259 access_key_id> XXXX (as shown when creating the access grant) 37260 Option secret_access_key. 37261 AWS Secret Access Key (password). 37262 Leave blank for anonymous access or runtime credentials. 37263 Enter a value. Press Enter to leave empty. 37264 secret_access_key> XXXX (as shown when creating the access grant) 37265 Option endpoint. 37266 Endpoint of the Shared Gateway. 37267 Choose a number from below, or type in your own value. 37268 Press Enter to leave empty. 37269 1 / EU1 Shared Gateway 37270 \[rs] (gateway.eu1.storjshare.io) 37271 2 / US1 Shared Gateway 37272 \[rs] (gateway.us1.storjshare.io) 37273 3 / Asia-Pacific Shared Gateway 37274 \[rs] (gateway.ap1.storjshare.io) 37275 endpoint> 1 (as shown when creating the access grant) 37276 Edit advanced config? 37277 y) Yes 37278 n) No (default) 37279 y/n> n 37280 \f[R] 37281 .fi 37282 .PP 37283 Note that s3 credentials are generated when you create an access 37284 grant (https://docs.storj.io/dcs/api-reference/s3-compatible-gateway#usage). 37285 .SS Backend quirks 37286 .IP \[bu] 2 37287 \f[C]--chunk-size\f[R] is forced to be 64 MiB or greater. 37288 This will use more memory than the default of 5 MiB. 37289 .IP \[bu] 2 37290 Server side copy is disabled as it isn\[aq]t currently supported in the 37291 gateway. 37292 .IP \[bu] 2 37293 GetTier and SetTier are not supported. 37294 .SS Backend bugs 37295 .PP 37296 Due to issue #39 (https://github.com/storj/gateway-mt/issues/39) 37297 uploading multipart files via the S3 gateway causes them to lose their 37298 metadata. 37299 For rclone\[aq]s purpose this means that the modification time is not 37300 stored, nor is any MD5SUM (if one is available from the source). 37301 .PP 37302 This has the following consequences: 37303 .IP \[bu] 2 37304 Using \f[C]rclone rcat\f[R] will fail as the medatada doesn\[aq]t match 37305 after upload 37306 .IP \[bu] 2 37307 Uploading files with \f[C]rclone mount\f[R] will fail for the same 37308 reason 37309 .RS 2 37310 .IP \[bu] 2 37311 This can worked around by using \f[C]--vfs-cache-mode writes\f[R] or 37312 \f[C]--vfs-cache-mode full\f[R] or setting \f[C]--s3-upload-cutoff\f[R] 37313 large 37314 .RE 37315 .IP \[bu] 2 37316 Files uploaded via a multipart upload won\[aq]t have their modtimes 37317 .RS 2 37318 .IP \[bu] 2 37319 This will mean that \f[C]rclone sync\f[R] will likely keep trying to 37320 upload files bigger than \f[C]--s3-upload-cutoff\f[R] 37321 .IP \[bu] 2 37322 This can be worked around with \f[C]--checksum\f[R] or 37323 \f[C]--size-only\f[R] or setting \f[C]--s3-upload-cutoff\f[R] large 37324 .IP \[bu] 2 37325 The maximum value for \f[C]--s3-upload-cutoff\f[R] is 5GiB though 37326 .RE 37327 .PP 37328 One general purpose workaround is to set 37329 \f[C]--s3-upload-cutoff 5G\f[R]. 37330 This means that rclone will upload files smaller than 5GiB as single 37331 parts. 37332 Note that this can be set in the config file with 37333 \f[C]upload_cutoff = 5G\f[R] or configured in the advanced settings. 37334 If you regularly transfer files larger than 5G then using 37335 \f[C]--checksum\f[R] or \f[C]--size-only\f[R] in \f[C]rclone sync\f[R] 37336 is the recommended workaround. 37337 .SS Comparison with the native protocol 37338 .PP 37339 Use the the native protocol to take advantage of client-side encryption 37340 as well as to achieve the best possible download performance. 37341 Uploads will be erasure-coded locally, thus a 1gb upload will result in 37342 2.68gb of data being uploaded to storage nodes across the network. 37343 .PP 37344 Use this backend and the S3 compatible Hosted Gateway to increase upload 37345 performance and reduce the load on your systems and network. 37346 Uploads will be encrypted and erasure-coded server-side, thus a 1GB 37347 upload will result in only in 1GB of data being uploaded to storage 37348 nodes across the network. 37349 .PP 37350 For more detailed comparison please check the documentation of the storj 37351 backend. 37352 .SS Limitations 37353 .PP 37354 \f[C]rclone about\f[R] is not supported by the S3 backend. 37355 Backends without this capability cannot determine free space for an 37356 rclone mount or use policy \f[C]mfs\f[R] (most free space) as a member 37357 of an rclone union remote. 37358 .PP 37359 See List of backends that do not support rclone 37360 about (https://rclone.org/overview/#optional-features) and rclone 37361 about (https://rclone.org/commands/rclone_about/) 37362 .SS Synology C2 Object Storage 37363 .PP 37364 Synology C2 Object 37365 Storage (https://c2.synology.com/en-global/object-storage/overview) 37366 provides a secure, S3-compatible, and cost-effective cloud storage 37367 solution without API request, download fees, and deletion penalty. 37368 .PP 37369 The S3 compatible gateway is configured using \f[C]rclone config\f[R] 37370 with a type of \f[C]s3\f[R] and with a provider name of 37371 \f[C]Synology\f[R]. 37372 Here is an example run of the configurator. 37373 .PP 37374 First run: 37375 .IP 37376 .nf 37377 \f[C] 37378 rclone config 37379 \f[R] 37380 .fi 37381 .PP 37382 This will guide you through an interactive setup process. 37383 .IP 37384 .nf 37385 \f[C] 37386 No remotes found, make a new one? 37387 n) New remote 37388 s) Set configuration password 37389 q) Quit config 37390 37391 n/s/q> n 37392 37393 Enter name for new remote.1 37394 name> syno 37395 37396 Type of storage to configure. 37397 Enter a string value. Press Enter for the default (\[dq]\[dq]). 37398 Choose a number from below, or type in your own value 37399 37400 XX / Amazon S3 Compliant Storage Providers including AWS, ... 37401 \[rs] \[dq]s3\[dq] 37402 37403 Storage> s3 37404 37405 Choose your S3 provider. 37406 Enter a string value. Press Enter for the default (\[dq]\[dq]). 37407 Choose a number from below, or type in your own value 37408 24 / Synology C2 Object Storage 37409 \[rs] (Synology) 37410 37411 provider> Synology 37412 37413 Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars). 37414 Only applies if access_key_id and secret_access_key is blank. 37415 Enter a boolean value (true or false). Press Enter for the default (\[dq]false\[dq]). 37416 Choose a number from below, or type in your own value 37417 1 / Enter AWS credentials in the next step 37418 \[rs] \[dq]false\[dq] 37419 2 / Get AWS credentials from the environment (env vars or IAM) 37420 \[rs] \[dq]true\[dq] 37421 37422 env_auth> 1 37423 37424 AWS Access Key ID. 37425 Leave blank for anonymous access or runtime credentials. 37426 Enter a string value. Press Enter for the default (\[dq]\[dq]). 37427 37428 access_key_id> accesskeyid 37429 37430 AWS Secret Access Key (password) 37431 Leave blank for anonymous access or runtime credentials. 37432 Enter a string value. Press Enter for the default (\[dq]\[dq]). 37433 37434 secret_access_key> secretaccesskey 37435 37436 Region where your data stored. 37437 Choose a number from below, or type in your own value. 37438 Press Enter to leave empty. 37439 1 / Europe Region 1 37440 \[rs] (eu-001) 37441 2 / Europe Region 2 37442 \[rs] (eu-002) 37443 3 / US Region 1 37444 \[rs] (us-001) 37445 4 / US Region 2 37446 \[rs] (us-002) 37447 5 / Asia (Taiwan) 37448 \[rs] (tw-001) 37449 37450 region > 1 37451 37452 Option endpoint. 37453 Endpoint for Synology C2 Object Storage API. 37454 Choose a number from below, or type in your own value. 37455 Press Enter to leave empty. 37456 1 / EU Endpoint 1 37457 \[rs] (eu-001.s3.synologyc2.net) 37458 2 / US Endpoint 1 37459 \[rs] (us-001.s3.synologyc2.net) 37460 3 / TW Endpoint 1 37461 \[rs] (tw-001.s3.synologyc2.net) 37462 37463 endpoint> 1 37464 37465 Option location_constraint. 37466 Location constraint - must be set to match the Region. 37467 Leave blank if not sure. Used when creating buckets only. 37468 Enter a value. Press Enter to leave empty. 37469 location_constraint> 37470 37471 Edit advanced config? (y/n) 37472 y) Yes 37473 n) No 37474 y/n> y 37475 37476 Option no_check_bucket. 37477 If set, don\[aq]t attempt to check the bucket exists or create it. 37478 This can be useful when trying to minimise the number of transactions 37479 rclone does if you know the bucket exists already. 37480 It can also be needed if the user you are using does not have bucket 37481 creation permissions. Before v1.52.0 this would have passed silently 37482 due to a bug. 37483 Enter a boolean value (true or false). Press Enter for the default (true). 37484 37485 no_check_bucket> true 37486 37487 Configuration complete. 37488 Options: 37489 - type: s3 37490 - provider: Synology 37491 - region: eu-001 37492 - endpoint: eu-001.s3.synologyc2.net 37493 - no_check_bucket: true 37494 Keep this \[dq]syno\[dq] remote? 37495 y) Yes this is OK (default) 37496 e) Edit this remote 37497 d) Delete this remote 37498 37499 y/e/d> y 37500 37501 # Backblaze B2 37502 37503 B2 is [Backblaze\[aq]s cloud storage system](https://www.backblaze.com/b2/). 37504 37505 Paths are specified as \[ga]remote:bucket\[ga] (or \[ga]remote:\[ga] for the \[ga]lsd\[ga] 37506 command.) You may put subdirectories in too, e.g. \[ga]remote:bucket/path/to/dir\[ga]. 37507 37508 ## Configuration 37509 37510 Here is an example of making a b2 configuration. First run 37511 37512 rclone config 37513 37514 This will guide you through an interactive setup process. To authenticate 37515 you will either need your Account ID (a short hex number) and Master 37516 Application Key (a long hex number) OR an Application Key, which is the 37517 recommended method. See below for further details on generating and using 37518 an Application Key. 37519 \f[R] 37520 .fi 37521 .PP 37522 No remotes found, make a new one? 37523 n) New remote q) Quit config n/q> n name> remote Type of storage to 37524 configure. 37525 Choose a number from below, or type in your own value [snip] XX / 37526 Backblaze B2 \ \[dq]b2\[dq] [snip] Storage> b2 Account ID or Application 37527 Key ID account> 123456789abc Application Key key> 37528 0123456789abcdef0123456789abcdef0123456789 Endpoint for the service - 37529 leave blank normally. 37530 endpoint> Remote config -------------------- [remote] account = 37531 123456789abc key = 0123456789abcdef0123456789abcdef0123456789 endpoint = 37532 -------------------- y) Yes this is OK e) Edit this remote d) Delete 37533 this remote y/e/d> y 37534 .IP 37535 .nf 37536 \f[C] 37537 This remote is called \[ga]remote\[ga] and can now be used like this 37538 37539 See all buckets 37540 37541 rclone lsd remote: 37542 37543 Create a new bucket 37544 37545 rclone mkdir remote:bucket 37546 37547 List the contents of a bucket 37548 37549 rclone ls remote:bucket 37550 37551 Sync \[ga]/home/local/directory\[ga] to the remote bucket, deleting any 37552 excess files in the bucket. 37553 37554 rclone sync --interactive /home/local/directory remote:bucket 37555 37556 ### Application Keys 37557 37558 B2 supports multiple [Application Keys for different access permission 37559 to B2 Buckets](https://www.backblaze.com/b2/docs/application_keys.html). 37560 37561 You can use these with rclone too; you will need to use rclone version 1.43 37562 or later. 37563 37564 Follow Backblaze\[aq]s docs to create an Application Key with the required 37565 permission and add the \[ga]applicationKeyId\[ga] as the \[ga]account\[ga] and the 37566 \[ga]Application Key\[ga] itself as the \[ga]key\[ga]. 37567 37568 Note that you must put the _applicationKeyId_ as the \[ga]account\[ga] \[en] you 37569 can\[aq]t use the master Account ID. If you try then B2 will return 401 37570 errors. 37571 37572 ### --fast-list 37573 37574 This remote supports \[ga]--fast-list\[ga] which allows you to use fewer 37575 transactions in exchange for more memory. See the [rclone 37576 docs](https://rclone.org/docs/#fast-list) for more details. 37577 37578 ### Modification times 37579 37580 The modification time is stored as metadata on the object as 37581 \[ga]X-Bz-Info-src_last_modified_millis\[ga] as milliseconds since 1970-01-01 37582 in the Backblaze standard. Other tools should be able to use this as 37583 a modified time. 37584 37585 Modified times are used in syncing and are fully supported. Note that 37586 if a modification time needs to be updated on an object then it will 37587 create a new version of the object. 37588 37589 ### Restricted filename characters 37590 37591 In addition to the [default restricted characters set](https://rclone.org/overview/#restricted-characters) 37592 the following characters are also replaced: 37593 37594 | Character | Value | Replacement | 37595 | --------- |:-----:|:-----------:| 37596 | \[rs] | 0x5C | \[uFF3C] | 37597 37598 Invalid UTF-8 bytes will also be [replaced](https://rclone.org/overview/#invalid-utf8), 37599 as they can\[aq]t be used in JSON strings. 37600 37601 Note that in 2020-05 Backblaze started allowing \[rs] characters in file 37602 names. Rclone hasn\[aq]t changed its encoding as this could cause syncs to 37603 re-transfer files. If you want rclone not to replace \[rs] then see the 37604 \[ga]--b2-encoding\[ga] flag below and remove the \[ga]BackSlash\[ga] from the 37605 string. This can be set in the config. 37606 37607 ### SHA1 checksums 37608 37609 The SHA1 checksums of the files are checked on upload and download and 37610 will be used in the syncing process. 37611 37612 Large files (bigger than the limit in \[ga]--b2-upload-cutoff\[ga]) which are 37613 uploaded in chunks will store their SHA1 on the object as 37614 \[ga]X-Bz-Info-large_file_sha1\[ga] as recommended by Backblaze. 37615 37616 For a large file to be uploaded with an SHA1 checksum, the source 37617 needs to support SHA1 checksums. The local disk supports SHA1 37618 checksums so large file transfers from local disk will have an SHA1. 37619 See [the overview](https://rclone.org/overview/#features) for exactly which remotes 37620 support SHA1. 37621 37622 Sources which don\[aq]t support SHA1, in particular \[ga]crypt\[ga] will upload 37623 large files without SHA1 checksums. This may be fixed in the future 37624 (see [#1767](https://github.com/artpar/artpar/issues/1767)). 37625 37626 Files sizes below \[ga]--b2-upload-cutoff\[ga] will always have an SHA1 37627 regardless of the source. 37628 37629 ### Transfers 37630 37631 Backblaze recommends that you do lots of transfers simultaneously for 37632 maximum speed. In tests from my SSD equipped laptop the optimum 37633 setting is about \[ga]--transfers 32\[ga] though higher numbers may be used 37634 for a slight speed improvement. The optimum number for you may vary 37635 depending on your hardware, how big the files are, how much you want 37636 to load your computer, etc. The default of \[ga]--transfers 4\[ga] is 37637 definitely too low for Backblaze B2 though. 37638 37639 Note that uploading big files (bigger than 200 MiB by default) will use 37640 a 96 MiB RAM buffer by default. There can be at most \[ga]--transfers\[ga] of 37641 these in use at any moment, so this sets the upper limit on the memory 37642 used. 37643 37644 ### Versions 37645 37646 When rclone uploads a new version of a file it creates a [new version 37647 of it](https://www.backblaze.com/b2/docs/file_versions.html). 37648 Likewise when you delete a file, the old version will be marked hidden 37649 and still be available. Conversely, you may opt in to a \[dq]hard delete\[dq] 37650 of files with the \[ga]--b2-hard-delete\[ga] flag which would permanently remove 37651 the file instead of hiding it. 37652 37653 Old versions of files, where available, are visible using the 37654 \[ga]--b2-versions\[ga] flag. 37655 37656 It is also possible to view a bucket as it was at a certain point in time, 37657 using the \[ga]--b2-version-at\[ga] flag. This will show the file versions as they 37658 were at that time, showing files that have been deleted afterwards, and 37659 hiding files that were created since. 37660 37661 If you wish to remove all the old versions then you can use the 37662 \[ga]rclone cleanup remote:bucket\[ga] command which will delete all the old 37663 versions of files, leaving the current ones intact. You can also 37664 supply a path and only old versions under that path will be deleted, 37665 e.g. \[ga]rclone cleanup remote:bucket/path/to/stuff\[ga]. 37666 37667 Note that \[ga]cleanup\[ga] will remove partially uploaded files from the bucket 37668 if they are more than a day old. 37669 37670 When you \[ga]purge\[ga] a bucket, the current and the old versions will be 37671 deleted then the bucket will be deleted. 37672 37673 However \[ga]delete\[ga] will cause the current versions of the files to 37674 become hidden old versions. 37675 37676 Here is a session showing the listing and retrieval of an old 37677 version followed by a \[ga]cleanup\[ga] of the old versions. 37678 37679 Show current version and all the versions with \[ga]--b2-versions\[ga] flag. 37680 \f[R] 37681 .fi 37682 .PP 37683 $ rclone -q ls b2:cleanup-test 9 one.txt 37684 .PP 37685 $ rclone -q --b2-versions ls b2:cleanup-test 9 one.txt 8 37686 one-v2016-07-04-141032-000.txt 16 one-v2016-07-04-141003-000.txt 15 37687 one-v2016-07-02-155621-000.txt 37688 .IP 37689 .nf 37690 \f[C] 37691 Retrieve an old version 37692 \f[R] 37693 .fi 37694 .PP 37695 $ rclone -q --b2-versions copy 37696 b2:cleanup-test/one-v2016-07-04-141003-000.txt /tmp 37697 .PP 37698 $ ls -l /tmp/one-v2016-07-04-141003-000.txt -rw-rw-r-- 1 ncw ncw 16 Jul 37699 2 17:46 /tmp/one-v2016-07-04-141003-000.txt 37700 .IP 37701 .nf 37702 \f[C] 37703 Clean up all the old versions and show that they\[aq]ve gone. 37704 \f[R] 37705 .fi 37706 .PP 37707 $ rclone -q cleanup b2:cleanup-test 37708 .PP 37709 $ rclone -q ls b2:cleanup-test 9 one.txt 37710 .PP 37711 $ rclone -q --b2-versions ls b2:cleanup-test 9 one.txt 37712 .IP 37713 .nf 37714 \f[C] 37715 #### Versions naming caveat 37716 37717 When using \[ga]--b2-versions\[ga] flag rclone is relying on the file name 37718 to work out whether the objects are versions or not. Versions\[aq] names 37719 are created by inserting timestamp between file name and its extension. 37720 \f[R] 37721 .fi 37722 .IP 37723 .nf 37724 \f[C] 37725 9 file.txt 37726 8 file-v2023-07-17-161032-000.txt 37727 16 file-v2023-06-15-141003-000.txt 37728 \f[R] 37729 .fi 37730 .IP 37731 .nf 37732 \f[C] 37733 If there are real files present with the same names as versions, then 37734 behaviour of \[ga]--b2-versions\[ga] can be unpredictable. 37735 37736 ### Data usage 37737 37738 It is useful to know how many requests are sent to the server in different scenarios. 37739 37740 All copy commands send the following 4 requests: 37741 \f[R] 37742 .fi 37743 .PP 37744 /b2api/v1/b2_authorize_account /b2api/v1/b2_create_bucket 37745 /b2api/v1/b2_list_buckets /b2api/v1/b2_list_file_names 37746 .IP 37747 .nf 37748 \f[C] 37749 The \[ga]b2_list_file_names\[ga] request will be sent once for every 1k files 37750 in the remote path, providing the checksum and modification time of 37751 the listed files. As of version 1.33 issue 37752 [#818](https://github.com/artpar/artpar/issues/818) causes extra requests 37753 to be sent when using B2 with Crypt. When a copy operation does not 37754 require any files to be uploaded, no more requests will be sent. 37755 37756 Uploading files that do not require chunking, will send 2 requests per 37757 file upload: 37758 \f[R] 37759 .fi 37760 .PP 37761 /b2api/v1/b2_get_upload_url /b2api/v1/b2_upload_file/ 37762 .IP 37763 .nf 37764 \f[C] 37765 Uploading files requiring chunking, will send 2 requests (one each to 37766 start and finish the upload) and another 2 requests for each chunk: 37767 \f[R] 37768 .fi 37769 .PP 37770 /b2api/v1/b2_start_large_file /b2api/v1/b2_get_upload_part_url 37771 /b2api/v1/b2_upload_part/ /b2api/v1/b2_finish_large_file 37772 .IP 37773 .nf 37774 \f[C] 37775 #### Versions 37776 37777 Versions can be viewed with the \[ga]--b2-versions\[ga] flag. When it is set 37778 rclone will show and act on older versions of files. For example 37779 37780 Listing without \[ga]--b2-versions\[ga] 37781 \f[R] 37782 .fi 37783 .PP 37784 $ rclone -q ls b2:cleanup-test 9 one.txt 37785 .IP 37786 .nf 37787 \f[C] 37788 And with 37789 \f[R] 37790 .fi 37791 .PP 37792 $ rclone -q --b2-versions ls b2:cleanup-test 9 one.txt 8 37793 one-v2016-07-04-141032-000.txt 16 one-v2016-07-04-141003-000.txt 15 37794 one-v2016-07-02-155621-000.txt 37795 .IP 37796 .nf 37797 \f[C] 37798 Showing that the current version is unchanged but older versions can 37799 be seen. These have the UTC date that they were uploaded to the 37800 server to the nearest millisecond appended to them. 37801 37802 Note that when using \[ga]--b2-versions\[ga] no file write operations are 37803 permitted, so you can\[aq]t upload files or delete them. 37804 37805 ### B2 and rclone link 37806 37807 Rclone supports generating file share links for private B2 buckets. 37808 They can either be for a file for example: 37809 \f[R] 37810 .fi 37811 .PP 37812 \&./rclone link B2:bucket/path/to/file.txt 37813 https://f002.backblazeb2.com/file/bucket/path/to/file.txt?Authorization=xxxxxxxx 37814 .IP 37815 .nf 37816 \f[C] 37817 or if run on a directory you will get: 37818 \f[R] 37819 .fi 37820 .PP 37821 \&./rclone link B2:bucket/path 37822 https://f002.backblazeb2.com/file/bucket/path?Authorization=xxxxxxxx 37823 .IP 37824 .nf 37825 \f[C] 37826 you can then use the authorization token (the part of the url from the 37827 \[ga]?Authorization=\[ga] on) on any file path under that directory. For example: 37828 \f[R] 37829 .fi 37830 .PP 37831 https://f002.backblazeb2.com/file/bucket/path/to/file1?Authorization=xxxxxxxx 37832 https://f002.backblazeb2.com/file/bucket/path/file2?Authorization=xxxxxxxx 37833 https://f002.backblazeb2.com/file/bucket/path/folder/file3?Authorization=xxxxxxxx 37834 .IP 37835 .nf 37836 \f[C] 37837 37838 ### Standard options 37839 37840 Here are the Standard options specific to b2 (Backblaze B2). 37841 37842 #### --b2-account 37843 37844 Account ID or Application Key ID. 37845 37846 Properties: 37847 37848 - Config: account 37849 - Env Var: RCLONE_B2_ACCOUNT 37850 - Type: string 37851 - Required: true 37852 37853 #### --b2-key 37854 37855 Application Key. 37856 37857 Properties: 37858 37859 - Config: key 37860 - Env Var: RCLONE_B2_KEY 37861 - Type: string 37862 - Required: true 37863 37864 #### --b2-hard-delete 37865 37866 Permanently delete files on remote removal, otherwise hide files. 37867 37868 Properties: 37869 37870 - Config: hard_delete 37871 - Env Var: RCLONE_B2_HARD_DELETE 37872 - Type: bool 37873 - Default: false 37874 37875 ### Advanced options 37876 37877 Here are the Advanced options specific to b2 (Backblaze B2). 37878 37879 #### --b2-endpoint 37880 37881 Endpoint for the service. 37882 37883 Leave blank normally. 37884 37885 Properties: 37886 37887 - Config: endpoint 37888 - Env Var: RCLONE_B2_ENDPOINT 37889 - Type: string 37890 - Required: false 37891 37892 #### --b2-test-mode 37893 37894 A flag string for X-Bz-Test-Mode header for debugging. 37895 37896 This is for debugging purposes only. Setting it to one of the strings 37897 below will cause b2 to return specific errors: 37898 37899 * \[dq]fail_some_uploads\[dq] 37900 * \[dq]expire_some_account_authorization_tokens\[dq] 37901 * \[dq]force_cap_exceeded\[dq] 37902 37903 These will be set in the \[dq]X-Bz-Test-Mode\[dq] header which is documented 37904 in the [b2 integrations checklist](https://www.backblaze.com/b2/docs/integration_checklist.html). 37905 37906 Properties: 37907 37908 - Config: test_mode 37909 - Env Var: RCLONE_B2_TEST_MODE 37910 - Type: string 37911 - Required: false 37912 37913 #### --b2-versions 37914 37915 Include old versions in directory listings. 37916 37917 Note that when using this no file write operations are permitted, 37918 so you can\[aq]t upload files or delete them. 37919 37920 Properties: 37921 37922 - Config: versions 37923 - Env Var: RCLONE_B2_VERSIONS 37924 - Type: bool 37925 - Default: false 37926 37927 #### --b2-version-at 37928 37929 Show file versions as they were at the specified time. 37930 37931 Note that when using this no file write operations are permitted, 37932 so you can\[aq]t upload files or delete them. 37933 37934 Properties: 37935 37936 - Config: version_at 37937 - Env Var: RCLONE_B2_VERSION_AT 37938 - Type: Time 37939 - Default: off 37940 37941 #### --b2-upload-cutoff 37942 37943 Cutoff for switching to chunked upload. 37944 37945 Files above this size will be uploaded in chunks of \[dq]--b2-chunk-size\[dq]. 37946 37947 This value should be set no larger than 4.657 GiB (== 5 GB). 37948 37949 Properties: 37950 37951 - Config: upload_cutoff 37952 - Env Var: RCLONE_B2_UPLOAD_CUTOFF 37953 - Type: SizeSuffix 37954 - Default: 200Mi 37955 37956 #### --b2-copy-cutoff 37957 37958 Cutoff for switching to multipart copy. 37959 37960 Any files larger than this that need to be server-side copied will be 37961 copied in chunks of this size. 37962 37963 The minimum is 0 and the maximum is 4.6 GiB. 37964 37965 Properties: 37966 37967 - Config: copy_cutoff 37968 - Env Var: RCLONE_B2_COPY_CUTOFF 37969 - Type: SizeSuffix 37970 - Default: 4Gi 37971 37972 #### --b2-chunk-size 37973 37974 Upload chunk size. 37975 37976 When uploading large files, chunk the file into this size. 37977 37978 Must fit in memory. These chunks are buffered in memory and there 37979 might a maximum of \[dq]--transfers\[dq] chunks in progress at once. 37980 37981 5,000,000 Bytes is the minimum size. 37982 37983 Properties: 37984 37985 - Config: chunk_size 37986 - Env Var: RCLONE_B2_CHUNK_SIZE 37987 - Type: SizeSuffix 37988 - Default: 96Mi 37989 37990 #### --b2-upload-concurrency 37991 37992 Concurrency for multipart uploads. 37993 37994 This is the number of chunks of the same file that are uploaded 37995 concurrently. 37996 37997 Note that chunks are stored in memory and there may be up to 37998 \[dq]--transfers\[dq] * \[dq]--b2-upload-concurrency\[dq] chunks stored at once 37999 in memory. 38000 38001 Properties: 38002 38003 - Config: upload_concurrency 38004 - Env Var: RCLONE_B2_UPLOAD_CONCURRENCY 38005 - Type: int 38006 - Default: 4 38007 38008 #### --b2-disable-checksum 38009 38010 Disable checksums for large (> upload cutoff) files. 38011 38012 Normally rclone will calculate the SHA1 checksum of the input before 38013 uploading it so it can add it to metadata on the object. This is great 38014 for data integrity checking but can cause long delays for large files 38015 to start uploading. 38016 38017 Properties: 38018 38019 - Config: disable_checksum 38020 - Env Var: RCLONE_B2_DISABLE_CHECKSUM 38021 - Type: bool 38022 - Default: false 38023 38024 #### --b2-download-url 38025 38026 Custom endpoint for downloads. 38027 38028 This is usually set to a Cloudflare CDN URL as Backblaze offers 38029 free egress for data downloaded through the Cloudflare network. 38030 Rclone works with private buckets by sending an \[dq]Authorization\[dq] header. 38031 If the custom endpoint rewrites the requests for authentication, 38032 e.g., in Cloudflare Workers, this header needs to be handled properly. 38033 Leave blank if you want to use the endpoint provided by Backblaze. 38034 38035 The URL provided here SHOULD have the protocol and SHOULD NOT have 38036 a trailing slash or specify the /file/bucket subpath as rclone will 38037 request files with \[dq]{download_url}/file/{bucket_name}/{path}\[dq]. 38038 38039 Example: 38040 > https://mysubdomain.mydomain.tld 38041 (No trailing \[dq]/\[dq], \[dq]file\[dq] or \[dq]bucket\[dq]) 38042 38043 Properties: 38044 38045 - Config: download_url 38046 - Env Var: RCLONE_B2_DOWNLOAD_URL 38047 - Type: string 38048 - Required: false 38049 38050 #### --b2-download-auth-duration 38051 38052 Time before the public link authorization token will expire in s or suffix ms|s|m|h|d. 38053 38054 This is used in combination with \[dq]rclone link\[dq] for making files 38055 accessible to the public and sets the duration before the download 38056 authorization token will expire. 38057 38058 The minimum value is 1 second. The maximum value is one week. 38059 38060 Properties: 38061 38062 - Config: download_auth_duration 38063 - Env Var: RCLONE_B2_DOWNLOAD_AUTH_DURATION 38064 - Type: Duration 38065 - Default: 1w 38066 38067 #### --b2-memory-pool-flush-time 38068 38069 How often internal memory buffer pools will be flushed. (no longer used) 38070 38071 Properties: 38072 38073 - Config: memory_pool_flush_time 38074 - Env Var: RCLONE_B2_MEMORY_POOL_FLUSH_TIME 38075 - Type: Duration 38076 - Default: 1m0s 38077 38078 #### --b2-memory-pool-use-mmap 38079 38080 Whether to use mmap buffers in internal memory pool. (no longer used) 38081 38082 Properties: 38083 38084 - Config: memory_pool_use_mmap 38085 - Env Var: RCLONE_B2_MEMORY_POOL_USE_MMAP 38086 - Type: bool 38087 - Default: false 38088 38089 #### --b2-lifecycle 38090 38091 Set the number of days deleted files should be kept when creating a bucket. 38092 38093 On bucket creation, this parameter is used to create a lifecycle rule 38094 for the entire bucket. 38095 38096 If lifecycle is 0 (the default) it does not create a lifecycle rule so 38097 the default B2 behaviour applies. This is to create versions of files 38098 on delete and overwrite and to keep them indefinitely. 38099 38100 If lifecycle is >0 then it creates a single rule setting the number of 38101 days before a file that is deleted or overwritten is deleted 38102 permanently. This is known as daysFromHidingToDeleting in the b2 docs. 38103 38104 The minimum value for this parameter is 1 day. 38105 38106 You can also enable hard_delete in the config also which will mean 38107 deletions won\[aq]t cause versions but overwrites will still cause 38108 versions to be made. 38109 38110 See: [rclone backend lifecycle](#lifecycle) for setting lifecycles after bucket creation. 38111 38112 38113 Properties: 38114 38115 - Config: lifecycle 38116 - Env Var: RCLONE_B2_LIFECYCLE 38117 - Type: int 38118 - Default: 0 38119 38120 #### --b2-encoding 38121 38122 The encoding for the backend. 38123 38124 See the [encoding section in the overview](https://rclone.org/overview/#encoding) for more info. 38125 38126 Properties: 38127 38128 - Config: encoding 38129 - Env Var: RCLONE_B2_ENCODING 38130 - Type: Encoding 38131 - Default: Slash,BackSlash,Del,Ctl,InvalidUtf8,Dot 38132 38133 #### --b2-description 38134 38135 Description of the remote 38136 38137 Properties: 38138 38139 - Config: description 38140 - Env Var: RCLONE_B2_DESCRIPTION 38141 - Type: string 38142 - Required: false 38143 38144 ## Backend commands 38145 38146 Here are the commands specific to the b2 backend. 38147 38148 Run them with 38149 38150 rclone backend COMMAND remote: 38151 38152 The help below will explain what arguments each command takes. 38153 38154 See the [backend](https://rclone.org/commands/rclone_backend/) command for more 38155 info on how to pass options and arguments. 38156 38157 These can be run on a running backend using the rc command 38158 [backend/command](https://rclone.org/rc/#backend-command). 38159 38160 ### lifecycle 38161 38162 Read or set the lifecycle for a bucket 38163 38164 rclone backend lifecycle remote: [options] [<arguments>+] 38165 38166 This command can be used to read or set the lifecycle for a bucket. 38167 38168 Usage Examples: 38169 38170 To show the current lifecycle rules: 38171 38172 rclone backend lifecycle b2:bucket 38173 38174 This will dump something like this showing the lifecycle rules. 38175 38176 [ 38177 { 38178 \[dq]daysFromHidingToDeleting\[dq]: 1, 38179 \[dq]daysFromUploadingToHiding\[dq]: null, 38180 \[dq]fileNamePrefix\[dq]: \[dq]\[dq] 38181 } 38182 ] 38183 38184 If there are no lifecycle rules (the default) then it will just return []. 38185 38186 To reset the current lifecycle rules: 38187 38188 rclone backend lifecycle b2:bucket -o daysFromHidingToDeleting=30 38189 rclone backend lifecycle b2:bucket -o daysFromUploadingToHiding=5 -o daysFromHidingToDeleting=1 38190 38191 This will run and then print the new lifecycle rules as above. 38192 38193 Rclone only lets you set lifecycles for the whole bucket with the 38194 fileNamePrefix = \[dq]\[dq]. 38195 38196 You can\[aq]t disable versioning with B2. The best you can do is to set 38197 the daysFromHidingToDeleting to 1 day. You can enable hard_delete in 38198 the config also which will mean deletions won\[aq]t cause versions but 38199 overwrites will still cause versions to be made. 38200 38201 rclone backend lifecycle b2:bucket -o daysFromHidingToDeleting=1 38202 38203 See: https://www.backblaze.com/docs/cloud-storage-lifecycle-rules 38204 38205 38206 Options: 38207 38208 - \[dq]daysFromHidingToDeleting\[dq]: After a file has been hidden for this many days it is deleted. 0 is off. 38209 - \[dq]daysFromUploadingToHiding\[dq]: This many days after uploading a file is hidden 38210 38211 38212 38213 ## Limitations 38214 38215 \[ga]rclone about\[ga] is not supported by the B2 backend. Backends without 38216 this capability cannot determine free space for an rclone mount or 38217 use policy \[ga]mfs\[ga] (most free space) as a member of an rclone union 38218 remote. 38219 38220 See [List of backends that do not support rclone about](https://rclone.org/overview/#optional-features) and [rclone about](https://rclone.org/commands/rclone_about/) 38221 38222 # Box 38223 38224 Paths are specified as \[ga]remote:path\[ga] 38225 38226 Paths may be as deep as required, e.g. \[ga]remote:directory/subdirectory\[ga]. 38227 38228 The initial setup for Box involves getting a token from Box which you 38229 can do either in your browser, or with a config.json downloaded from Box 38230 to use JWT authentication. \[ga]rclone config\[ga] walks you through it. 38231 38232 ## Configuration 38233 38234 Here is an example of how to make a remote called \[ga]remote\[ga]. First run: 38235 38236 rclone config 38237 38238 This will guide you through an interactive setup process: 38239 \f[R] 38240 .fi 38241 .PP 38242 No remotes found, make a new one? 38243 n) New remote s) Set configuration password q) Quit config n/s/q> n 38244 name> remote Type of storage to configure. 38245 Choose a number from below, or type in your own value [snip] XX / Box 38246 \ \[dq]box\[dq] [snip] Storage> box Box App Client Id - leave blank 38247 normally. 38248 client_id> Box App Client Secret - leave blank normally. 38249 client_secret> Box App config.json location Leave blank normally. 38250 Enter a string value. 38251 Press Enter for the default (\[dq]\[dq]). 38252 box_config_file> Box App Primary Access Token Leave blank normally. 38253 Enter a string value. 38254 Press Enter for the default (\[dq]\[dq]). 38255 access_token> 38256 .PP 38257 Enter a string value. 38258 Press Enter for the default (\[dq]user\[dq]). 38259 Choose a number from below, or type in your own value 1 / Rclone should 38260 act on behalf of a user \ \[dq]user\[dq] 2 / Rclone should act on behalf 38261 of a service account \ \[dq]enterprise\[dq] box_sub_type> Remote config 38262 Use web browser to automatically authenticate rclone with remote? 38263 * Say Y if the machine running rclone has a web browser you can use * 38264 Say N if running rclone on a (remote) machine without web browser access 38265 If not sure try Y. 38266 If Y failed, try N. 38267 y) Yes n) No y/n> y If your browser doesn\[aq]t open automatically go to 38268 the following link: http://127.0.0.1:53682/auth Log in and authorize 38269 rclone for access Waiting for code... 38270 Got code -------------------- [remote] client_id = client_secret = token 38271 = 38272 {\[dq]access_token\[dq]:\[dq]XXX\[dq],\[dq]token_type\[dq]:\[dq]bearer\[dq],\[dq]refresh_token\[dq]:\[dq]XXX\[dq],\[dq]expiry\[dq]:\[dq]XXX\[dq]} 38273 -------------------- y) Yes this is OK e) Edit this remote d) Delete 38274 this remote y/e/d> y 38275 .IP 38276 .nf 38277 \f[C] 38278 See the [remote setup docs](https://rclone.org/remote_setup/) for how to set it up on a 38279 machine with no Internet browser available. 38280 38281 Note that rclone runs a webserver on your local machine to collect the 38282 token as returned from Box. This only runs from the moment it opens 38283 your browser to the moment you get back the verification code. This 38284 is on \[ga]http://127.0.0.1:53682/\[ga] and this it may require you to unblock 38285 it temporarily if you are running a host firewall. 38286 38287 Once configured you can then use \[ga]rclone\[ga] like this, 38288 38289 List directories in top level of your Box 38290 38291 rclone lsd remote: 38292 38293 List all the files in your Box 38294 38295 rclone ls remote: 38296 38297 To copy a local directory to an Box directory called backup 38298 38299 rclone copy /home/source remote:backup 38300 38301 ### Using rclone with an Enterprise account with SSO 38302 38303 If you have an \[dq]Enterprise\[dq] account type with Box with single sign on 38304 (SSO), you need to create a password to use Box with rclone. This can 38305 be done at your Enterprise Box account by going to Settings, \[dq]Account\[dq] 38306 Tab, and then set the password in the \[dq]Authentication\[dq] field. 38307 38308 Once you have done this, you can setup your Enterprise Box account 38309 using the same procedure detailed above in the, using the password you 38310 have just set. 38311 38312 ### Invalid refresh token 38313 38314 According to the [box docs](https://developer.box.com/v2.0/docs/oauth-20#section-6-using-the-access-and-refresh-tokens): 38315 38316 > Each refresh_token is valid for one use in 60 days. 38317 38318 This means that if you 38319 38320 * Don\[aq]t use the box remote for 60 days 38321 * Copy the config file with a box refresh token in and use it in two places 38322 * Get an error on a token refresh 38323 38324 then rclone will return an error which includes the text \[ga]Invalid 38325 refresh token\[ga]. 38326 38327 To fix this you will need to use oauth2 again to update the refresh 38328 token. You can use the methods in [the remote setup 38329 docs](https://rclone.org/remote_setup/), bearing in mind that if you use the copy the 38330 config file method, you should not use that remote on the computer you 38331 did the authentication on. 38332 38333 Here is how to do it. 38334 \f[R] 38335 .fi 38336 .PP 38337 $ rclone config Current remotes: 38338 .PP 38339 Name Type ==== ==== remote box 38340 .IP "e)" 3 38341 Edit existing remote 38342 .IP "f)" 3 38343 New remote 38344 .IP "g)" 3 38345 Delete remote 38346 .IP "h)" 3 38347 Rename remote 38348 .IP "i)" 3 38349 Copy remote 38350 .IP "j)" 3 38351 Set configuration password 38352 .IP "k)" 3 38353 Quit config e/n/d/r/c/s/q> e Choose a number from below, or type in an 38354 existing value 1 > remote remote> remote -------------------- [remote] 38355 type = box token = 38356 {\[dq]access_token\[dq]:\[dq]XXX\[dq],\[dq]token_type\[dq]:\[dq]bearer\[dq],\[dq]refresh_token\[dq]:\[dq]XXX\[dq],\[dq]expiry\[dq]:\[dq]2017-07-08T23:40:08.059167677+01:00\[dq]} 38357 -------------------- Edit remote Value \[dq]client_id\[dq] = \[dq]\[dq] 38358 Edit? 38359 (y/n)> 38360 .IP "l)" 3 38361 Yes 38362 .IP "m)" 3 38363 No y/n> n Value \[dq]client_secret\[dq] = \[dq]\[dq] Edit? 38364 (y/n)> 38365 .IP "n)" 3 38366 Yes 38367 .IP "o)" 3 38368 No y/n> n Remote config Already have a token - refresh? 38369 .IP "p)" 3 38370 Yes 38371 .IP "q)" 3 38372 No y/n> y Use web browser to automatically authenticate rclone with 38373 remote? 38374 .IP \[bu] 2 38375 Say Y if the machine running rclone has a web browser you can use 38376 .IP \[bu] 2 38377 Say N if running rclone on a (remote) machine without web browser access 38378 If not sure try Y. 38379 If Y failed, try N. 38380 .IP "y)" 3 38381 Yes 38382 .IP "z)" 3 38383 No y/n> y If your browser doesn\[aq]t open automatically go to the 38384 following link: http://127.0.0.1:53682/auth Log in and authorize rclone 38385 for access Waiting for code... 38386 Got code -------------------- [remote] type = box token = 38387 {\[dq]access_token\[dq]:\[dq]YYY\[dq],\[dq]token_type\[dq]:\[dq]bearer\[dq],\[dq]refresh_token\[dq]:\[dq]YYY\[dq],\[dq]expiry\[dq]:\[dq]2017-07-23T12:22:29.259137901+01:00\[dq]} 38388 -------------------- 38389 .IP "a)" 3 38390 Yes this is OK 38391 .IP "b)" 3 38392 Edit this remote 38393 .IP "c)" 3 38394 Delete this remote y/e/d> y 38395 .IP 38396 .nf 38397 \f[C] 38398 ### Modification times and hashes 38399 38400 Box allows modification times to be set on objects accurate to 1 38401 second. These will be used to detect whether objects need syncing or 38402 not. 38403 38404 Box supports SHA1 type hashes, so you can use the \[ga]--checksum\[ga] 38405 flag. 38406 38407 ### Restricted filename characters 38408 38409 In addition to the [default restricted characters set](https://rclone.org/overview/#restricted-characters) 38410 the following characters are also replaced: 38411 38412 | Character | Value | Replacement | 38413 | --------- |:-----:|:-----------:| 38414 | \[rs] | 0x5C | \[uFF3C] | 38415 38416 File names can also not end with the following characters. 38417 These only get replaced if they are the last character in the name: 38418 38419 | Character | Value | Replacement | 38420 | --------- |:-----:|:-----------:| 38421 | SP | 0x20 | \[u2420] | 38422 38423 Invalid UTF-8 bytes will also be [replaced](https://rclone.org/overview/#invalid-utf8), 38424 as they can\[aq]t be used in JSON strings. 38425 38426 ### Transfers 38427 38428 For files above 50 MiB rclone will use a chunked transfer. Rclone will 38429 upload up to \[ga]--transfers\[ga] chunks at the same time (shared among all 38430 the multipart uploads). Chunks are buffered in memory and are 38431 normally 8 MiB so increasing \[ga]--transfers\[ga] will increase memory use. 38432 38433 ### Deleting files 38434 38435 Depending on the enterprise settings for your user, the item will 38436 either be actually deleted from Box or moved to the trash. 38437 38438 Emptying the trash is supported via the rclone however cleanup command 38439 however this deletes every trashed file and folder individually so it 38440 may take a very long time. 38441 Emptying the trash via the WebUI does not have this limitation 38442 so it is advised to empty the trash via the WebUI. 38443 38444 ### Root folder ID 38445 38446 You can set the \[ga]root_folder_id\[ga] for rclone. This is the directory 38447 (identified by its \[ga]Folder ID\[ga]) that rclone considers to be the root 38448 of your Box drive. 38449 38450 Normally you will leave this blank and rclone will determine the 38451 correct root to use itself. 38452 38453 However you can set this to restrict rclone to a specific folder 38454 hierarchy. 38455 38456 In order to do this you will have to find the \[ga]Folder ID\[ga] of the 38457 directory you wish rclone to display. This will be the last segment 38458 of the URL when you open the relevant folder in the Box web 38459 interface. 38460 38461 So if the folder you want rclone to use has a URL which looks like 38462 \[ga]https://app.box.com/folder/11xxxxxxxxx8\[ga] 38463 in the browser, then you use \[ga]11xxxxxxxxx8\[ga] as 38464 the \[ga]root_folder_id\[ga] in the config. 38465 38466 38467 ### Standard options 38468 38469 Here are the Standard options specific to box (Box). 38470 38471 #### --box-client-id 38472 38473 OAuth Client Id. 38474 38475 Leave blank normally. 38476 38477 Properties: 38478 38479 - Config: client_id 38480 - Env Var: RCLONE_BOX_CLIENT_ID 38481 - Type: string 38482 - Required: false 38483 38484 #### --box-client-secret 38485 38486 OAuth Client Secret. 38487 38488 Leave blank normally. 38489 38490 Properties: 38491 38492 - Config: client_secret 38493 - Env Var: RCLONE_BOX_CLIENT_SECRET 38494 - Type: string 38495 - Required: false 38496 38497 #### --box-box-config-file 38498 38499 Box App config.json location 38500 38501 Leave blank normally. 38502 38503 Leading \[ga]\[ti]\[ga] will be expanded in the file name as will environment variables such as \[ga]${RCLONE_CONFIG_DIR}\[ga]. 38504 38505 Properties: 38506 38507 - Config: box_config_file 38508 - Env Var: RCLONE_BOX_BOX_CONFIG_FILE 38509 - Type: string 38510 - Required: false 38511 38512 #### --box-access-token 38513 38514 Box App Primary Access Token 38515 38516 Leave blank normally. 38517 38518 Properties: 38519 38520 - Config: access_token 38521 - Env Var: RCLONE_BOX_ACCESS_TOKEN 38522 - Type: string 38523 - Required: false 38524 38525 #### --box-box-sub-type 38526 38527 38528 38529 Properties: 38530 38531 - Config: box_sub_type 38532 - Env Var: RCLONE_BOX_BOX_SUB_TYPE 38533 - Type: string 38534 - Default: \[dq]user\[dq] 38535 - Examples: 38536 - \[dq]user\[dq] 38537 - Rclone should act on behalf of a user. 38538 - \[dq]enterprise\[dq] 38539 - Rclone should act on behalf of a service account. 38540 38541 ### Advanced options 38542 38543 Here are the Advanced options specific to box (Box). 38544 38545 #### --box-token 38546 38547 OAuth Access Token as a JSON blob. 38548 38549 Properties: 38550 38551 - Config: token 38552 - Env Var: RCLONE_BOX_TOKEN 38553 - Type: string 38554 - Required: false 38555 38556 #### --box-auth-url 38557 38558 Auth server URL. 38559 38560 Leave blank to use the provider defaults. 38561 38562 Properties: 38563 38564 - Config: auth_url 38565 - Env Var: RCLONE_BOX_AUTH_URL 38566 - Type: string 38567 - Required: false 38568 38569 #### --box-token-url 38570 38571 Token server url. 38572 38573 Leave blank to use the provider defaults. 38574 38575 Properties: 38576 38577 - Config: token_url 38578 - Env Var: RCLONE_BOX_TOKEN_URL 38579 - Type: string 38580 - Required: false 38581 38582 #### --box-root-folder-id 38583 38584 Fill in for rclone to use a non root folder as its starting point. 38585 38586 Properties: 38587 38588 - Config: root_folder_id 38589 - Env Var: RCLONE_BOX_ROOT_FOLDER_ID 38590 - Type: string 38591 - Default: \[dq]0\[dq] 38592 38593 #### --box-upload-cutoff 38594 38595 Cutoff for switching to multipart upload (>= 50 MiB). 38596 38597 Properties: 38598 38599 - Config: upload_cutoff 38600 - Env Var: RCLONE_BOX_UPLOAD_CUTOFF 38601 - Type: SizeSuffix 38602 - Default: 50Mi 38603 38604 #### --box-commit-retries 38605 38606 Max number of times to try committing a multipart file. 38607 38608 Properties: 38609 38610 - Config: commit_retries 38611 - Env Var: RCLONE_BOX_COMMIT_RETRIES 38612 - Type: int 38613 - Default: 100 38614 38615 #### --box-list-chunk 38616 38617 Size of listing chunk 1-1000. 38618 38619 Properties: 38620 38621 - Config: list_chunk 38622 - Env Var: RCLONE_BOX_LIST_CHUNK 38623 - Type: int 38624 - Default: 1000 38625 38626 #### --box-owned-by 38627 38628 Only show items owned by the login (email address) passed in. 38629 38630 Properties: 38631 38632 - Config: owned_by 38633 - Env Var: RCLONE_BOX_OWNED_BY 38634 - Type: string 38635 - Required: false 38636 38637 #### --box-impersonate 38638 38639 Impersonate this user ID when using a service account. 38640 38641 Setting this flag allows rclone, when using a JWT service account, to 38642 act on behalf of another user by setting the as-user header. 38643 38644 The user ID is the Box identifier for a user. User IDs can found for 38645 any user via the GET /users endpoint, which is only available to 38646 admins, or by calling the GET /users/me endpoint with an authenticated 38647 user session. 38648 38649 See: https://developer.box.com/guides/authentication/jwt/as-user/ 38650 38651 38652 Properties: 38653 38654 - Config: impersonate 38655 - Env Var: RCLONE_BOX_IMPERSONATE 38656 - Type: string 38657 - Required: false 38658 38659 #### --box-encoding 38660 38661 The encoding for the backend. 38662 38663 See the [encoding section in the overview](https://rclone.org/overview/#encoding) for more info. 38664 38665 Properties: 38666 38667 - Config: encoding 38668 - Env Var: RCLONE_BOX_ENCODING 38669 - Type: Encoding 38670 - Default: Slash,BackSlash,Del,Ctl,RightSpace,InvalidUtf8,Dot 38671 38672 #### --box-description 38673 38674 Description of the remote 38675 38676 Properties: 38677 38678 - Config: description 38679 - Env Var: RCLONE_BOX_DESCRIPTION 38680 - Type: string 38681 - Required: false 38682 38683 38684 38685 ## Limitations 38686 38687 Note that Box is case insensitive so you can\[aq]t have a file called 38688 \[dq]Hello.doc\[dq] and one called \[dq]hello.doc\[dq]. 38689 38690 Box file names can\[aq]t have the \[ga]\[rs]\[ga] character in. rclone maps this to 38691 and from an identical looking unicode equivalent \[ga]\[uFF3C]\[ga] (U+FF3C Fullwidth 38692 Reverse Solidus). 38693 38694 Box only supports filenames up to 255 characters in length. 38695 38696 Box has [API rate limits](https://developer.box.com/guides/api-calls/permissions-and-errors/rate-limits/) that sometimes reduce the speed of rclone. 38697 38698 \[ga]rclone about\[ga] is not supported by the Box backend. Backends without 38699 this capability cannot determine free space for an rclone mount or 38700 use policy \[ga]mfs\[ga] (most free space) as a member of an rclone union 38701 remote. 38702 38703 See [List of backends that do not support rclone about](https://rclone.org/overview/#optional-features) and [rclone about](https://rclone.org/commands/rclone_about/) 38704 38705 ## Get your own Box App ID 38706 38707 Here is how to create your own Box App ID for rclone: 38708 38709 1. Go to the [Box Developer Console](https://app.box.com/developers/console) 38710 and login, then click \[ga]My Apps\[ga] on the sidebar. Click \[ga]Create New App\[ga] 38711 and select \[ga]Custom App\[ga]. 38712 38713 2. In the first screen on the box that pops up, you can pretty much enter 38714 whatever you want. The \[ga]App Name\[ga] can be whatever. For \[ga]Purpose\[ga] choose 38715 automation to avoid having to fill out anything else. Click \[ga]Next\[ga]. 38716 38717 3. In the second screen of the creation screen, select 38718 \[ga]User Authentication (OAuth 2.0)\[ga]. Then click \[ga]Create App\[ga]. 38719 38720 4. You should now be on the \[ga]Configuration\[ga] tab of your new app. If not, 38721 click on it at the top of the webpage. Copy down \[ga]Client ID\[ga] 38722 and \[ga]Client Secret\[ga], you\[aq]ll need those for rclone. 38723 38724 5. Under \[dq]OAuth 2.0 Redirect URI\[dq], add \[ga]http://127.0.0.1:53682/\[ga] 38725 38726 6. For \[ga]Application Scopes\[ga], select \[ga]Read all files and folders stored in Box\[ga] 38727 and \[ga]Write all files and folders stored in box\[ga] (assuming you want to do both). 38728 Leave others unchecked. Click \[ga]Save Changes\[ga] at the top right. 38729 38730 # Cache 38731 38732 The \[ga]cache\[ga] remote wraps another existing remote and stores file structure 38733 and its data for long running tasks like \[ga]rclone mount\[ga]. 38734 38735 ## Status 38736 38737 The cache backend code is working but it currently doesn\[aq]t 38738 have a maintainer so there are [outstanding bugs](https://github.com/artpar/artpar/issues?q=is%3Aopen+is%3Aissue+label%3Abug+label%3A%22Remote%3A+Cache%22) which aren\[aq]t getting fixed. 38739 38740 The cache backend is due to be phased out in favour of the VFS caching 38741 layer eventually which is more tightly integrated into rclone. 38742 38743 Until this happens we recommend only using the cache backend if you 38744 find you can\[aq]t work without it. There are many docs online describing 38745 the use of the cache backend to minimize API hits and by-and-large 38746 these are out of date and the cache backend isn\[aq]t needed in those 38747 scenarios any more. 38748 38749 ## Configuration 38750 38751 To get started you just need to have an existing remote which can be configured 38752 with \[ga]cache\[ga]. 38753 38754 Here is an example of how to make a remote called \[ga]test-cache\[ga]. First run: 38755 38756 rclone config 38757 38758 This will guide you through an interactive setup process: 38759 \f[R] 38760 .fi 38761 .PP 38762 No remotes found, make a new one? 38763 n) New remote r) Rename remote c) Copy remote s) Set configuration 38764 password q) Quit config n/r/c/s/q> n name> test-cache Type of storage to 38765 configure. 38766 Choose a number from below, or type in your own value [snip] XX / Cache 38767 a remote \ \[dq]cache\[dq] [snip] Storage> cache Remote to cache. 38768 Normally should contain a \[aq]:\[aq] and a path, e.g. 38769 \[dq]myremote:path/to/dir\[dq], \[dq]myremote:bucket\[dq] or maybe 38770 \[dq]myremote:\[dq] (not recommended). 38771 remote> local:/test Optional: The URL of the Plex server plex_url> 38772 http://127.0.0.1:32400 Optional: The username of the Plex user 38773 plex_username> dummyusername Optional: The password of the Plex user y) 38774 Yes type in my own password g) Generate random password n) No leave this 38775 optional password blank y/g/n> y Enter the password: password: Confirm 38776 the password: password: The size of a chunk. 38777 Lower value good for slow connections but can affect seamless reading. 38778 Default: 5M Choose a number from below, or type in your own value 1 / 1 38779 MiB \ \[dq]1M\[dq] 2 / 5 MiB \ \[dq]5M\[dq] 3 / 10 MiB \ \[dq]10M\[dq] 38780 chunk_size> 2 How much time should object info (file size, file hashes, 38781 etc.) be stored in cache. 38782 Use a very high value if you don\[aq]t plan on changing the source FS 38783 from outside the cache. 38784 Accepted units are: \[dq]s\[dq], \[dq]m\[dq], \[dq]h\[dq]. 38785 Default: 5m Choose a number from below, or type in your own value 1 / 1 38786 hour \ \[dq]1h\[dq] 2 / 24 hours \ \[dq]24h\[dq] 3 / 24 hours 38787 \ \[dq]48h\[dq] info_age> 2 The maximum size of stored chunks. 38788 When the storage grows beyond this size, the oldest chunks will be 38789 deleted. 38790 Default: 10G Choose a number from below, or type in your own value 1 / 38791 500 MiB \ \[dq]500M\[dq] 2 / 1 GiB \ \[dq]1G\[dq] 3 / 10 GiB 38792 \ \[dq]10G\[dq] chunk_total_size> 3 Remote config -------------------- 38793 [test-cache] remote = local:/test plex_url = http://127.0.0.1:32400 38794 plex_username = dummyusername plex_password = *** ENCRYPTED *** 38795 chunk_size = 5M info_age = 48h chunk_total_size = 10G 38796 .IP 38797 .nf 38798 \f[C] 38799 You can then use it like this, 38800 38801 List directories in top level of your drive 38802 38803 rclone lsd test-cache: 38804 38805 List all the files in your drive 38806 38807 rclone ls test-cache: 38808 38809 To start a cached mount 38810 38811 rclone mount --allow-other test-cache: /var/tmp/test-cache 38812 38813 ### Write Features ### 38814 38815 ### Offline uploading ### 38816 38817 In an effort to make writing through cache more reliable, the backend 38818 now supports this feature which can be activated by specifying a 38819 \[ga]cache-tmp-upload-path\[ga]. 38820 38821 A files goes through these states when using this feature: 38822 38823 1. An upload is started (usually by copying a file on the cache remote) 38824 2. When the copy to the temporary location is complete the file is part 38825 of the cached remote and looks and behaves like any other file (reading included) 38826 3. After \[ga]cache-tmp-wait-time\[ga] passes and the file is next in line, \[ga]rclone move\[ga] 38827 is used to move the file to the cloud provider 38828 4. Reading the file still works during the upload but most modifications on it will be prohibited 38829 5. Once the move is complete the file is unlocked for modifications as it 38830 becomes as any other regular file 38831 6. If the file is being read through \[ga]cache\[ga] when it\[aq]s actually 38832 deleted from the temporary path then \[ga]cache\[ga] will simply swap the source 38833 to the cloud provider without interrupting the reading (small blip can happen though) 38834 38835 Files are uploaded in sequence and only one file is uploaded at a time. 38836 Uploads will be stored in a queue and be processed based on the order they were added. 38837 The queue and the temporary storage is persistent across restarts but 38838 can be cleared on startup with the \[ga]--cache-db-purge\[ga] flag. 38839 38840 ### Write Support ### 38841 38842 Writes are supported through \[ga]cache\[ga]. 38843 One caveat is that a mounted cache remote does not add any retry or fallback 38844 mechanism to the upload operation. This will depend on the implementation 38845 of the wrapped remote. Consider using \[ga]Offline uploading\[ga] for reliable writes. 38846 38847 One special case is covered with \[ga]cache-writes\[ga] which will cache the file 38848 data at the same time as the upload when it is enabled making it available 38849 from the cache store immediately once the upload is finished. 38850 38851 ### Read Features ### 38852 38853 #### Multiple connections #### 38854 38855 To counter the high latency between a local PC where rclone is running 38856 and cloud providers, the cache remote can split multiple requests to the 38857 cloud provider for smaller file chunks and combines them together locally 38858 where they can be available almost immediately before the reader usually 38859 needs them. 38860 38861 This is similar to buffering when media files are played online. Rclone 38862 will stay around the current marker but always try its best to stay ahead 38863 and prepare the data before. 38864 38865 #### Plex Integration #### 38866 38867 There is a direct integration with Plex which allows cache to detect during reading 38868 if the file is in playback or not. This helps cache to adapt how it queries 38869 the cloud provider depending on what is needed for. 38870 38871 Scans will have a minimum amount of workers (1) while in a confirmed playback cache 38872 will deploy the configured number of workers. 38873 38874 This integration opens the doorway to additional performance improvements 38875 which will be explored in the near future. 38876 38877 **Note:** If Plex options are not configured, \[ga]cache\[ga] will function with its 38878 configured options without adapting any of its settings. 38879 38880 How to enable? Run \[ga]rclone config\[ga] and add all the Plex options (endpoint, username 38881 and password) in your remote and it will be automatically enabled. 38882 38883 Affected settings: 38884 - \[ga]cache-workers\[ga]: _Configured value_ during confirmed playback or _1_ all the other times 38885 38886 ##### Certificate Validation ##### 38887 38888 When the Plex server is configured to only accept secure connections, it is 38889 possible to use \[ga].plex.direct\[ga] URLs to ensure certificate validation succeeds. 38890 These URLs are used by Plex internally to connect to the Plex server securely. 38891 38892 The format for these URLs is the following: 38893 38894 \[ga]https://ip-with-dots-replaced.server-hash.plex.direct:32400/\[ga] 38895 38896 The \[ga]ip-with-dots-replaced\[ga] part can be any IPv4 address, where the dots 38897 have been replaced with dashes, e.g. \[ga]127.0.0.1\[ga] becomes \[ga]127-0-0-1\[ga]. 38898 38899 To get the \[ga]server-hash\[ga] part, the easiest way is to visit 38900 38901 https://plex.tv/api/resources?includeHttps=1&X-Plex-Token=your-plex-token 38902 38903 This page will list all the available Plex servers for your account 38904 with at least one \[ga].plex.direct\[ga] link for each. Copy one URL and replace 38905 the IP address with the desired address. This can be used as the 38906 \[ga]plex_url\[ga] value. 38907 38908 ### Known issues ### 38909 38910 #### Mount and --dir-cache-time #### 38911 38912 --dir-cache-time controls the first layer of directory caching which works at the mount layer. 38913 Being an independent caching mechanism from the \[ga]cache\[ga] backend, it will manage its own entries 38914 based on the configured time. 38915 38916 To avoid getting in a scenario where dir cache has obsolete data and cache would have the correct 38917 one, try to set \[ga]--dir-cache-time\[ga] to a lower time than \[ga]--cache-info-age\[ga]. Default values are 38918 already configured in this way. 38919 38920 #### Windows support - Experimental #### 38921 38922 There are a couple of issues with Windows \[ga]mount\[ga] functionality that still require some investigations. 38923 It should be considered as experimental thus far as fixes come in for this OS. 38924 38925 Most of the issues seem to be related to the difference between filesystems 38926 on Linux flavors and Windows as cache is heavily dependent on them. 38927 38928 Any reports or feedback on how cache behaves on this OS is greatly appreciated. 38929 38930 - https://github.com/artpar/artpar/issues/1935 38931 - https://github.com/artpar/artpar/issues/1907 38932 - https://github.com/artpar/artpar/issues/1834 38933 38934 #### Risk of throttling #### 38935 38936 Future iterations of the cache backend will make use of the pooling functionality 38937 of the cloud provider to synchronize and at the same time make writing through it 38938 more tolerant to failures. 38939 38940 There are a couple of enhancements in track to add these but in the meantime 38941 there is a valid concern that the expiring cache listings can lead to cloud provider 38942 throttles or bans due to repeated queries on it for very large mounts. 38943 38944 Some recommendations: 38945 - don\[aq]t use a very small interval for entry information (\[ga]--cache-info-age\[ga]) 38946 - while writes aren\[aq]t yet optimised, you can still write through \[ga]cache\[ga] which gives you the advantage 38947 of adding the file in the cache at the same time if configured to do so. 38948 38949 Future enhancements: 38950 38951 - https://github.com/artpar/artpar/issues/1937 38952 - https://github.com/artpar/artpar/issues/1936 38953 38954 #### cache and crypt #### 38955 38956 One common scenario is to keep your data encrypted in the cloud provider 38957 using the \[ga]crypt\[ga] remote. \[ga]crypt\[ga] uses a similar technique to wrap around 38958 an existing remote and handles this translation in a seamless way. 38959 38960 There is an issue with wrapping the remotes in this order: 38961 **cloud remote** -> **crypt** -> **cache** 38962 38963 During testing, I experienced a lot of bans with the remotes in this order. 38964 I suspect it might be related to how crypt opens files on the cloud provider 38965 which makes it think we\[aq]re downloading the full file instead of small chunks. 38966 Organizing the remotes in this order yields better results: 38967 **cloud remote** -> **cache** -> **crypt** 38968 38969 #### absolute remote paths #### 38970 38971 \[ga]cache\[ga] can not differentiate between relative and absolute paths for the wrapped remote. 38972 Any path given in the \[ga]remote\[ga] config setting and on the command line will be passed to 38973 the wrapped remote as is, but for storing the chunks on disk the path will be made 38974 relative by removing any leading \[ga]/\[ga] character. 38975 38976 This behavior is irrelevant for most backend types, but there are backends where a leading \[ga]/\[ga] 38977 changes the effective directory, e.g. in the \[ga]sftp\[ga] backend paths starting with a \[ga]/\[ga] are 38978 relative to the root of the SSH server and paths without are relative to the user home directory. 38979 As a result \[ga]sftp:bin\[ga] and \[ga]sftp:/bin\[ga] will share the same cache folder, even if they represent 38980 a different directory on the SSH server. 38981 38982 ### Cache and Remote Control (--rc) ### 38983 Cache supports the new \[ga]--rc\[ga] mode in rclone and can be remote controlled through the following end points: 38984 By default, the listener is disabled if you do not add the flag. 38985 38986 ### rc cache/expire 38987 Purge a remote from the cache backend. Supports either a directory or a file. 38988 It supports both encrypted and unencrypted file names if cache is wrapped by crypt. 38989 38990 Params: 38991 - **remote** = path to remote **(required)** 38992 - **withData** = true/false to delete cached data (chunks) as well _(optional, false by default)_ 38993 38994 38995 ### Standard options 38996 38997 Here are the Standard options specific to cache (Cache a remote). 38998 38999 #### --cache-remote 39000 39001 Remote to cache. 39002 39003 Normally should contain a \[aq]:\[aq] and a path, e.g. \[dq]myremote:path/to/dir\[dq], 39004 \[dq]myremote:bucket\[dq] or maybe \[dq]myremote:\[dq] (not recommended). 39005 39006 Properties: 39007 39008 - Config: remote 39009 - Env Var: RCLONE_CACHE_REMOTE 39010 - Type: string 39011 - Required: true 39012 39013 #### --cache-plex-url 39014 39015 The URL of the Plex server. 39016 39017 Properties: 39018 39019 - Config: plex_url 39020 - Env Var: RCLONE_CACHE_PLEX_URL 39021 - Type: string 39022 - Required: false 39023 39024 #### --cache-plex-username 39025 39026 The username of the Plex user. 39027 39028 Properties: 39029 39030 - Config: plex_username 39031 - Env Var: RCLONE_CACHE_PLEX_USERNAME 39032 - Type: string 39033 - Required: false 39034 39035 #### --cache-plex-password 39036 39037 The password of the Plex user. 39038 39039 **NB** Input to this must be obscured - see [rclone obscure](https://rclone.org/commands/rclone_obscure/). 39040 39041 Properties: 39042 39043 - Config: plex_password 39044 - Env Var: RCLONE_CACHE_PLEX_PASSWORD 39045 - Type: string 39046 - Required: false 39047 39048 #### --cache-chunk-size 39049 39050 The size of a chunk (partial file data). 39051 39052 Use lower numbers for slower connections. If the chunk size is 39053 changed, any downloaded chunks will be invalid and cache-chunk-path 39054 will need to be cleared or unexpected EOF errors will occur. 39055 39056 Properties: 39057 39058 - Config: chunk_size 39059 - Env Var: RCLONE_CACHE_CHUNK_SIZE 39060 - Type: SizeSuffix 39061 - Default: 5Mi 39062 - Examples: 39063 - \[dq]1M\[dq] 39064 - 1 MiB 39065 - \[dq]5M\[dq] 39066 - 5 MiB 39067 - \[dq]10M\[dq] 39068 - 10 MiB 39069 39070 #### --cache-info-age 39071 39072 How long to cache file structure information (directory listings, file size, times, etc.). 39073 If all write operations are done through the cache then you can safely make 39074 this value very large as the cache store will also be updated in real time. 39075 39076 Properties: 39077 39078 - Config: info_age 39079 - Env Var: RCLONE_CACHE_INFO_AGE 39080 - Type: Duration 39081 - Default: 6h0m0s 39082 - Examples: 39083 - \[dq]1h\[dq] 39084 - 1 hour 39085 - \[dq]24h\[dq] 39086 - 24 hours 39087 - \[dq]48h\[dq] 39088 - 48 hours 39089 39090 #### --cache-chunk-total-size 39091 39092 The total size that the chunks can take up on the local disk. 39093 39094 If the cache exceeds this value then it will start to delete the 39095 oldest chunks until it goes under this value. 39096 39097 Properties: 39098 39099 - Config: chunk_total_size 39100 - Env Var: RCLONE_CACHE_CHUNK_TOTAL_SIZE 39101 - Type: SizeSuffix 39102 - Default: 10Gi 39103 - Examples: 39104 - \[dq]500M\[dq] 39105 - 500 MiB 39106 - \[dq]1G\[dq] 39107 - 1 GiB 39108 - \[dq]10G\[dq] 39109 - 10 GiB 39110 39111 ### Advanced options 39112 39113 Here are the Advanced options specific to cache (Cache a remote). 39114 39115 #### --cache-plex-token 39116 39117 The plex token for authentication - auto set normally. 39118 39119 Properties: 39120 39121 - Config: plex_token 39122 - Env Var: RCLONE_CACHE_PLEX_TOKEN 39123 - Type: string 39124 - Required: false 39125 39126 #### --cache-plex-insecure 39127 39128 Skip all certificate verification when connecting to the Plex server. 39129 39130 Properties: 39131 39132 - Config: plex_insecure 39133 - Env Var: RCLONE_CACHE_PLEX_INSECURE 39134 - Type: string 39135 - Required: false 39136 39137 #### --cache-db-path 39138 39139 Directory to store file structure metadata DB. 39140 39141 The remote name is used as the DB file name. 39142 39143 Properties: 39144 39145 - Config: db_path 39146 - Env Var: RCLONE_CACHE_DB_PATH 39147 - Type: string 39148 - Default: \[dq]$HOME/.cache/rclone/cache-backend\[dq] 39149 39150 #### --cache-chunk-path 39151 39152 Directory to cache chunk files. 39153 39154 Path to where partial file data (chunks) are stored locally. The remote 39155 name is appended to the final path. 39156 39157 This config follows the \[dq]--cache-db-path\[dq]. If you specify a custom 39158 location for \[dq]--cache-db-path\[dq] and don\[aq]t specify one for \[dq]--cache-chunk-path\[dq] 39159 then \[dq]--cache-chunk-path\[dq] will use the same path as \[dq]--cache-db-path\[dq]. 39160 39161 Properties: 39162 39163 - Config: chunk_path 39164 - Env Var: RCLONE_CACHE_CHUNK_PATH 39165 - Type: string 39166 - Default: \[dq]$HOME/.cache/rclone/cache-backend\[dq] 39167 39168 #### --cache-db-purge 39169 39170 Clear all the cached data for this remote on start. 39171 39172 Properties: 39173 39174 - Config: db_purge 39175 - Env Var: RCLONE_CACHE_DB_PURGE 39176 - Type: bool 39177 - Default: false 39178 39179 #### --cache-chunk-clean-interval 39180 39181 How often should the cache perform cleanups of the chunk storage. 39182 39183 The default value should be ok for most people. If you find that the 39184 cache goes over \[dq]cache-chunk-total-size\[dq] too often then try to lower 39185 this value to force it to perform cleanups more often. 39186 39187 Properties: 39188 39189 - Config: chunk_clean_interval 39190 - Env Var: RCLONE_CACHE_CHUNK_CLEAN_INTERVAL 39191 - Type: Duration 39192 - Default: 1m0s 39193 39194 #### --cache-read-retries 39195 39196 How many times to retry a read from a cache storage. 39197 39198 Since reading from a cache stream is independent from downloading file 39199 data, readers can get to a point where there\[aq]s no more data in the 39200 cache. Most of the times this can indicate a connectivity issue if 39201 cache isn\[aq]t able to provide file data anymore. 39202 39203 For really slow connections, increase this to a point where the stream is 39204 able to provide data but your experience will be very stuttering. 39205 39206 Properties: 39207 39208 - Config: read_retries 39209 - Env Var: RCLONE_CACHE_READ_RETRIES 39210 - Type: int 39211 - Default: 10 39212 39213 #### --cache-workers 39214 39215 How many workers should run in parallel to download chunks. 39216 39217 Higher values will mean more parallel processing (better CPU needed) 39218 and more concurrent requests on the cloud provider. This impacts 39219 several aspects like the cloud provider API limits, more stress on the 39220 hardware that rclone runs on but it also means that streams will be 39221 more fluid and data will be available much more faster to readers. 39222 39223 **Note**: If the optional Plex integration is enabled then this 39224 setting will adapt to the type of reading performed and the value 39225 specified here will be used as a maximum number of workers to use. 39226 39227 Properties: 39228 39229 - Config: workers 39230 - Env Var: RCLONE_CACHE_WORKERS 39231 - Type: int 39232 - Default: 4 39233 39234 #### --cache-chunk-no-memory 39235 39236 Disable the in-memory cache for storing chunks during streaming. 39237 39238 By default, cache will keep file data during streaming in RAM as well 39239 to provide it to readers as fast as possible. 39240 39241 This transient data is evicted as soon as it is read and the number of 39242 chunks stored doesn\[aq]t exceed the number of workers. However, depending 39243 on other settings like \[dq]cache-chunk-size\[dq] and \[dq]cache-workers\[dq] this footprint 39244 can increase if there are parallel streams too (multiple files being read 39245 at the same time). 39246 39247 If the hardware permits it, use this feature to provide an overall better 39248 performance during streaming but it can also be disabled if RAM is not 39249 available on the local machine. 39250 39251 Properties: 39252 39253 - Config: chunk_no_memory 39254 - Env Var: RCLONE_CACHE_CHUNK_NO_MEMORY 39255 - Type: bool 39256 - Default: false 39257 39258 #### --cache-rps 39259 39260 Limits the number of requests per second to the source FS (-1 to disable). 39261 39262 This setting places a hard limit on the number of requests per second 39263 that cache will be doing to the cloud provider remote and try to 39264 respect that value by setting waits between reads. 39265 39266 If you find that you\[aq]re getting banned or limited on the cloud 39267 provider through cache and know that a smaller number of requests per 39268 second will allow you to work with it then you can use this setting 39269 for that. 39270 39271 A good balance of all the other settings should make this setting 39272 useless but it is available to set for more special cases. 39273 39274 **NOTE**: This will limit the number of requests during streams but 39275 other API calls to the cloud provider like directory listings will 39276 still pass. 39277 39278 Properties: 39279 39280 - Config: rps 39281 - Env Var: RCLONE_CACHE_RPS 39282 - Type: int 39283 - Default: -1 39284 39285 #### --cache-writes 39286 39287 Cache file data on writes through the FS. 39288 39289 If you need to read files immediately after you upload them through 39290 cache you can enable this flag to have their data stored in the 39291 cache store at the same time during upload. 39292 39293 Properties: 39294 39295 - Config: writes 39296 - Env Var: RCLONE_CACHE_WRITES 39297 - Type: bool 39298 - Default: false 39299 39300 #### --cache-tmp-upload-path 39301 39302 Directory to keep temporary files until they are uploaded. 39303 39304 This is the path where cache will use as a temporary storage for new 39305 files that need to be uploaded to the cloud provider. 39306 39307 Specifying a value will enable this feature. Without it, it is 39308 completely disabled and files will be uploaded directly to the cloud 39309 provider 39310 39311 Properties: 39312 39313 - Config: tmp_upload_path 39314 - Env Var: RCLONE_CACHE_TMP_UPLOAD_PATH 39315 - Type: string 39316 - Required: false 39317 39318 #### --cache-tmp-wait-time 39319 39320 How long should files be stored in local cache before being uploaded. 39321 39322 This is the duration that a file must wait in the temporary location 39323 _cache-tmp-upload-path_ before it is selected for upload. 39324 39325 Note that only one file is uploaded at a time and it can take longer 39326 to start the upload if a queue formed for this purpose. 39327 39328 Properties: 39329 39330 - Config: tmp_wait_time 39331 - Env Var: RCLONE_CACHE_TMP_WAIT_TIME 39332 - Type: Duration 39333 - Default: 15s 39334 39335 #### --cache-db-wait-time 39336 39337 How long to wait for the DB to be available - 0 is unlimited. 39338 39339 Only one process can have the DB open at any one time, so rclone waits 39340 for this duration for the DB to become available before it gives an 39341 error. 39342 39343 If you set it to 0 then it will wait forever. 39344 39345 Properties: 39346 39347 - Config: db_wait_time 39348 - Env Var: RCLONE_CACHE_DB_WAIT_TIME 39349 - Type: Duration 39350 - Default: 1s 39351 39352 #### --cache-description 39353 39354 Description of the remote 39355 39356 Properties: 39357 39358 - Config: description 39359 - Env Var: RCLONE_CACHE_DESCRIPTION 39360 - Type: string 39361 - Required: false 39362 39363 ## Backend commands 39364 39365 Here are the commands specific to the cache backend. 39366 39367 Run them with 39368 39369 rclone backend COMMAND remote: 39370 39371 The help below will explain what arguments each command takes. 39372 39373 See the [backend](https://rclone.org/commands/rclone_backend/) command for more 39374 info on how to pass options and arguments. 39375 39376 These can be run on a running backend using the rc command 39377 [backend/command](https://rclone.org/rc/#backend-command). 39378 39379 ### stats 39380 39381 Print stats on the cache backend in JSON format. 39382 39383 rclone backend stats remote: [options] [<arguments>+] 39384 39385 39386 39387 # Chunker 39388 39389 The \[ga]chunker\[ga] overlay transparently splits large files into smaller chunks 39390 during upload to wrapped remote and transparently assembles them back 39391 when the file is downloaded. This allows to effectively overcome size limits 39392 imposed by storage providers. 39393 39394 ## Configuration 39395 39396 To use it, first set up the underlying remote following the configuration 39397 instructions for that remote. You can also use a local pathname instead of 39398 a remote. 39399 39400 First check your chosen remote is working - we\[aq]ll call it \[ga]remote:path\[ga] here. 39401 Note that anything inside \[ga]remote:path\[ga] will be chunked and anything outside 39402 won\[aq]t. This means that if you are using a bucket-based remote (e.g. S3, B2, swift) 39403 then you should probably put the bucket in the remote \[ga]s3:bucket\[ga]. 39404 39405 Now configure \[ga]chunker\[ga] using \[ga]rclone config\[ga]. We will call this one \[ga]overlay\[ga] 39406 to separate it from the \[ga]remote\[ga] itself. 39407 \f[R] 39408 .fi 39409 .PP 39410 No remotes found, make a new one? 39411 n) New remote s) Set configuration password q) Quit config n/s/q> n 39412 name> overlay Type of storage to configure. 39413 Choose a number from below, or type in your own value [snip] XX / 39414 Transparently chunk/split large files \ \[dq]chunker\[dq] [snip] 39415 Storage> chunker Remote to chunk/unchunk. 39416 Normally should contain a \[aq]:\[aq] and a path, e.g. 39417 \[dq]myremote:path/to/dir\[dq], \[dq]myremote:bucket\[dq] or maybe 39418 \[dq]myremote:\[dq] (not recommended). 39419 Enter a string value. 39420 Press Enter for the default (\[dq]\[dq]). 39421 remote> remote:path Files larger than chunk size will be split in 39422 chunks. 39423 Enter a size with suffix K,M,G,T. 39424 Press Enter for the default (\[dq]2G\[dq]). 39425 chunk_size> 100M Choose how chunker handles hash sums. 39426 All modes but \[dq]none\[dq] require metadata. 39427 Enter a string value. 39428 Press Enter for the default (\[dq]md5\[dq]). 39429 Choose a number from below, or type in your own value 1 / Pass any hash 39430 supported by wrapped remote for non-chunked files, return nothing 39431 otherwise \ \[dq]none\[dq] 2 / MD5 for composite files \ \[dq]md5\[dq] 3 39432 / SHA1 for composite files \ \[dq]sha1\[dq] 4 / MD5 for all files 39433 \ \[dq]md5all\[dq] 5 / SHA1 for all files \ \[dq]sha1all\[dq] 6 / 39434 Copying a file to chunker will request MD5 from the source falling back 39435 to SHA1 if unsupported \ \[dq]md5quick\[dq] 7 / Similar to 39436 \[dq]md5quick\[dq] but prefers SHA1 over MD5 \ \[dq]sha1quick\[dq] 39437 hash_type> md5 Edit advanced config? 39438 (y/n) y) Yes n) No y/n> n Remote config -------------------- [overlay] 39439 type = chunker remote = remote:bucket chunk_size = 100M hash_type = md5 39440 -------------------- y) Yes this is OK e) Edit this remote d) Delete 39441 this remote y/e/d> y 39442 .IP 39443 .nf 39444 \f[C] 39445 ### Specifying the remote 39446 39447 In normal use, make sure the remote has a \[ga]:\[ga] in. If you specify the remote 39448 without a \[ga]:\[ga] then rclone will use a local directory of that name. 39449 So if you use a remote of \[ga]/path/to/secret/files\[ga] then rclone will 39450 chunk stuff in that directory. If you use a remote of \[ga]name\[ga] then rclone 39451 will put files in a directory called \[ga]name\[ga] in the current directory. 39452 39453 39454 ### Chunking 39455 39456 When rclone starts a file upload, chunker checks the file size. If it 39457 doesn\[aq]t exceed the configured chunk size, chunker will just pass the file 39458 to the wrapped remote (however, see caveat below). If a file is large, chunker will transparently cut 39459 data in pieces with temporary names and stream them one by one, on the fly. 39460 Each data chunk will contain the specified number of bytes, except for the 39461 last one which may have less data. If file size is unknown in advance 39462 (this is called a streaming upload), chunker will internally create 39463 a temporary copy, record its size and repeat the above process. 39464 39465 When upload completes, temporary chunk files are finally renamed. 39466 This scheme guarantees that operations can be run in parallel and look 39467 from outside as atomic. 39468 A similar method with hidden temporary chunks is used for other operations 39469 (copy/move/rename, etc.). If an operation fails, hidden chunks are normally 39470 destroyed, and the target composite file stays intact. 39471 39472 When a composite file download is requested, chunker transparently 39473 assembles it by concatenating data chunks in order. As the split is trivial 39474 one could even manually concatenate data chunks together to obtain the 39475 original content. 39476 39477 When the \[ga]list\[ga] rclone command scans a directory on wrapped remote, 39478 the potential chunk files are accounted for, grouped and assembled into 39479 composite directory entries. Any temporary chunks are hidden. 39480 39481 List and other commands can sometimes come across composite files with 39482 missing or invalid chunks, e.g. shadowed by like-named directory or 39483 another file. This usually means that wrapped file system has been directly 39484 tampered with or damaged. If chunker detects a missing chunk it will 39485 by default print warning, skip the whole incomplete group of chunks but 39486 proceed with current command. 39487 You can set the \[ga]--chunker-fail-hard\[ga] flag to have commands abort with 39488 error message in such cases. 39489 39490 **Caveat**: As it is now, chunker will always create a temporary file in the 39491 backend and then rename it, even if the file is below the chunk threshold. 39492 This will result in unnecessary API calls and can severely restrict throughput 39493 when handling transfers primarily composed of small files on some backends (e.g. Box). 39494 A workaround to this issue is to use chunker only for files above the chunk threshold 39495 via \[ga]--min-size\[ga] and then perform a separate call without chunker on the remaining 39496 files. 39497 39498 39499 #### Chunk names 39500 39501 The default chunk name format is \[ga]*.rclone_chunk.###\[ga], hence by default 39502 chunk names are \[ga]BIG_FILE_NAME.rclone_chunk.001\[ga], 39503 \[ga]BIG_FILE_NAME.rclone_chunk.002\[ga] etc. You can configure another name format 39504 using the \[ga]name_format\[ga] configuration file option. The format uses asterisk 39505 \[ga]*\[ga] as a placeholder for the base file name and one or more consecutive 39506 hash characters \[ga]#\[ga] as a placeholder for sequential chunk number. 39507 There must be one and only one asterisk. The number of consecutive hash 39508 characters defines the minimum length of a string representing a chunk number. 39509 If decimal chunk number has less digits than the number of hashes, it is 39510 left-padded by zeros. If the decimal string is longer, it is left intact. 39511 By default numbering starts from 1 but there is another option that allows 39512 user to start from 0, e.g. for compatibility with legacy software. 39513 39514 For example, if name format is \[ga]big_*-##.part\[ga] and original file name is 39515 \[ga]data.txt\[ga] and numbering starts from 0, then the first chunk will be named 39516 \[ga]big_data.txt-00.part\[ga], the 99th chunk will be \[ga]big_data.txt-98.part\[ga] 39517 and the 302nd chunk will become \[ga]big_data.txt-301.part\[ga]. 39518 39519 Note that \[ga]list\[ga] assembles composite directory entries only when chunk names 39520 match the configured format and treats non-conforming file names as normal 39521 non-chunked files. 39522 39523 When using \[ga]norename\[ga] transactions, chunk names will additionally have a unique 39524 file version suffix. For example, \[ga]BIG_FILE_NAME.rclone_chunk.001_bp562k\[ga]. 39525 39526 39527 ### Metadata 39528 39529 Besides data chunks chunker will by default create metadata object for 39530 a composite file. The object is named after the original file. 39531 Chunker allows user to disable metadata completely (the \[ga]none\[ga] format). 39532 Note that metadata is normally not created for files smaller than the 39533 configured chunk size. This may change in future rclone releases. 39534 39535 #### Simple JSON metadata format 39536 39537 This is the default format. It supports hash sums and chunk validation 39538 for composite files. Meta objects carry the following fields: 39539 39540 - \[ga]ver\[ga] - version of format, currently \[ga]1\[ga] 39541 - \[ga]size\[ga] - total size of composite file 39542 - \[ga]nchunks\[ga] - number of data chunks in file 39543 - \[ga]md5\[ga] - MD5 hashsum of composite file (if present) 39544 - \[ga]sha1\[ga] - SHA1 hashsum (if present) 39545 - \[ga]txn\[ga] - identifies current version of the file 39546 39547 There is no field for composite file name as it\[aq]s simply equal to the name 39548 of meta object on the wrapped remote. Please refer to respective sections 39549 for details on hashsums and modified time handling. 39550 39551 #### No metadata 39552 39553 You can disable meta objects by setting the meta format option to \[ga]none\[ga]. 39554 In this mode chunker will scan directory for all files that follow 39555 configured chunk name format, group them by detecting chunks with the same 39556 base name and show group names as virtual composite files. 39557 This method is more prone to missing chunk errors (especially missing 39558 last chunk) than format with metadata enabled. 39559 39560 39561 ### Hashsums 39562 39563 Chunker supports hashsums only when a compatible metadata is present. 39564 Hence, if you choose metadata format of \[ga]none\[ga], chunker will report hashsum 39565 as \[ga]UNSUPPORTED\[ga]. 39566 39567 Please note that by default metadata is stored only for composite files. 39568 If a file is smaller than configured chunk size, chunker will transparently 39569 redirect hash requests to wrapped remote, so support depends on that. 39570 You will see the empty string as a hashsum of requested type for small 39571 files if the wrapped remote doesn\[aq]t support it. 39572 39573 Many storage backends support MD5 and SHA1 hash types, so does chunker. 39574 With chunker you can choose one or another but not both. 39575 MD5 is set by default as the most supported type. 39576 Since chunker keeps hashes for composite files and falls back to the 39577 wrapped remote hash for non-chunked ones, we advise you to choose the same 39578 hash type as supported by wrapped remote so that your file listings 39579 look coherent. 39580 39581 If your storage backend does not support MD5 or SHA1 but you need consistent 39582 file hashing, configure chunker with \[ga]md5all\[ga] or \[ga]sha1all\[ga]. These two modes 39583 guarantee given hash for all files. If wrapped remote doesn\[aq]t support it, 39584 chunker will then add metadata to all files, even small. However, this can 39585 double the amount of small files in storage and incur additional service charges. 39586 You can even use chunker to force md5/sha1 support in any other remote 39587 at expense of sidecar meta objects by setting e.g. \[ga]hash_type=sha1all\[ga] 39588 to force hashsums and \[ga]chunk_size=1P\[ga] to effectively disable chunking. 39589 39590 Normally, when a file is copied to chunker controlled remote, chunker 39591 will ask the file source for compatible file hash and revert to on-the-fly 39592 calculation if none is found. This involves some CPU overhead but provides 39593 a guarantee that given hashsum is available. Also, chunker will reject 39594 a server-side copy or move operation if source and destination hashsum 39595 types are different resulting in the extra network bandwidth, too. 39596 In some rare cases this may be undesired, so chunker provides two optional 39597 choices: \[ga]sha1quick\[ga] and \[ga]md5quick\[ga]. If the source does not support primary 39598 hash type and the quick mode is enabled, chunker will try to fall back to 39599 the secondary type. This will save CPU and bandwidth but can result in empty 39600 hashsums at destination. Beware of consequences: the \[ga]sync\[ga] command will 39601 revert (sometimes silently) to time/size comparison if compatible hashsums 39602 between source and target are not found. 39603 39604 39605 ### Modification times 39606 39607 Chunker stores modification times using the wrapped remote so support 39608 depends on that. For a small non-chunked file the chunker overlay simply 39609 manipulates modification time of the wrapped remote file. 39610 For a composite file with metadata chunker will get and set 39611 modification time of the metadata object on the wrapped remote. 39612 If file is chunked but metadata format is \[ga]none\[ga] then chunker will 39613 use modification time of the first data chunk. 39614 39615 39616 ### Migrations 39617 39618 The idiomatic way to migrate to a different chunk size, hash type, transaction 39619 style or chunk naming scheme is to: 39620 39621 - Collect all your chunked files under a directory and have your 39622 chunker remote point to it. 39623 - Create another directory (most probably on the same cloud storage) 39624 and configure a new remote with desired metadata format, 39625 hash type, chunk naming etc. 39626 - Now run \[ga]rclone sync --interactive oldchunks: newchunks:\[ga] and all your data 39627 will be transparently converted in transfer. 39628 This may take some time, yet chunker will try server-side 39629 copy if possible. 39630 - After checking data integrity you may remove configuration section 39631 of the old remote. 39632 39633 If rclone gets killed during a long operation on a big composite file, 39634 hidden temporary chunks may stay in the directory. They will not be 39635 shown by the \[ga]list\[ga] command but will eat up your account quota. 39636 Please note that the \[ga]deletefile\[ga] command deletes only active 39637 chunks of a file. As a workaround, you can use remote of the wrapped 39638 file system to see them. 39639 An easy way to get rid of hidden garbage is to copy littered directory 39640 somewhere using the chunker remote and purge the original directory. 39641 The \[ga]copy\[ga] command will copy only active chunks while the \[ga]purge\[ga] will 39642 remove everything including garbage. 39643 39644 39645 ### Caveats and Limitations 39646 39647 Chunker requires wrapped remote to support server-side \[ga]move\[ga] (or \[ga]copy\[ga] + 39648 \[ga]delete\[ga]) operations, otherwise it will explicitly refuse to start. 39649 This is because it internally renames temporary chunk files to their final 39650 names when an operation completes successfully. 39651 39652 Chunker encodes chunk number in file name, so with default \[ga]name_format\[ga] 39653 setting it adds 17 characters. Also chunker adds 7 characters of temporary 39654 suffix during operations. Many file systems limit base file name without path 39655 by 255 characters. Using rclone\[aq]s crypt remote as a base file system limits 39656 file name by 143 characters. Thus, maximum name length is 231 for most files 39657 and 119 for chunker-over-crypt. A user in need can change name format to 39658 e.g. \[ga]*.rcc##\[ga] and save 10 characters (provided at most 99 chunks per file). 39659 39660 Note that a move implemented using the copy-and-delete method may incur 39661 double charging with some cloud storage providers. 39662 39663 Chunker will not automatically rename existing chunks when you run 39664 \[ga]rclone config\[ga] on a live remote and change the chunk name format. 39665 Beware that in result of this some files which have been treated as chunks 39666 before the change can pop up in directory listings as normal files 39667 and vice versa. The same warning holds for the chunk size. 39668 If you desperately need to change critical chunking settings, you should 39669 run data migration as described above. 39670 39671 If wrapped remote is case insensitive, the chunker overlay will inherit 39672 that property (so you can\[aq]t have a file called \[dq]Hello.doc\[dq] and \[dq]hello.doc\[dq] 39673 in the same directory). 39674 39675 Chunker included in rclone releases up to \[ga]v1.54\[ga] can sometimes fail to 39676 detect metadata produced by recent versions of rclone. We recommend users 39677 to keep rclone up-to-date to avoid data corruption. 39678 39679 Changing \[ga]transactions\[ga] is dangerous and requires explicit migration. 39680 39681 39682 ### Standard options 39683 39684 Here are the Standard options specific to chunker (Transparently chunk/split large files). 39685 39686 #### --chunker-remote 39687 39688 Remote to chunk/unchunk. 39689 39690 Normally should contain a \[aq]:\[aq] and a path, e.g. \[dq]myremote:path/to/dir\[dq], 39691 \[dq]myremote:bucket\[dq] or maybe \[dq]myremote:\[dq] (not recommended). 39692 39693 Properties: 39694 39695 - Config: remote 39696 - Env Var: RCLONE_CHUNKER_REMOTE 39697 - Type: string 39698 - Required: true 39699 39700 #### --chunker-chunk-size 39701 39702 Files larger than chunk size will be split in chunks. 39703 39704 Properties: 39705 39706 - Config: chunk_size 39707 - Env Var: RCLONE_CHUNKER_CHUNK_SIZE 39708 - Type: SizeSuffix 39709 - Default: 2Gi 39710 39711 #### --chunker-hash-type 39712 39713 Choose how chunker handles hash sums. 39714 39715 All modes but \[dq]none\[dq] require metadata. 39716 39717 Properties: 39718 39719 - Config: hash_type 39720 - Env Var: RCLONE_CHUNKER_HASH_TYPE 39721 - Type: string 39722 - Default: \[dq]md5\[dq] 39723 - Examples: 39724 - \[dq]none\[dq] 39725 - Pass any hash supported by wrapped remote for non-chunked files. 39726 - Return nothing otherwise. 39727 - \[dq]md5\[dq] 39728 - MD5 for composite files. 39729 - \[dq]sha1\[dq] 39730 - SHA1 for composite files. 39731 - \[dq]md5all\[dq] 39732 - MD5 for all files. 39733 - \[dq]sha1all\[dq] 39734 - SHA1 for all files. 39735 - \[dq]md5quick\[dq] 39736 - Copying a file to chunker will request MD5 from the source. 39737 - Falling back to SHA1 if unsupported. 39738 - \[dq]sha1quick\[dq] 39739 - Similar to \[dq]md5quick\[dq] but prefers SHA1 over MD5. 39740 39741 ### Advanced options 39742 39743 Here are the Advanced options specific to chunker (Transparently chunk/split large files). 39744 39745 #### --chunker-name-format 39746 39747 String format of chunk file names. 39748 39749 The two placeholders are: base file name (*) and chunk number (#...). 39750 There must be one and only one asterisk and one or more consecutive hash characters. 39751 If chunk number has less digits than the number of hashes, it is left-padded by zeros. 39752 If there are more digits in the number, they are left as is. 39753 Possible chunk files are ignored if their name does not match given format. 39754 39755 Properties: 39756 39757 - Config: name_format 39758 - Env Var: RCLONE_CHUNKER_NAME_FORMAT 39759 - Type: string 39760 - Default: \[dq]*.rclone_chunk.###\[dq] 39761 39762 #### --chunker-start-from 39763 39764 Minimum valid chunk number. Usually 0 or 1. 39765 39766 By default chunk numbers start from 1. 39767 39768 Properties: 39769 39770 - Config: start_from 39771 - Env Var: RCLONE_CHUNKER_START_FROM 39772 - Type: int 39773 - Default: 1 39774 39775 #### --chunker-meta-format 39776 39777 Format of the metadata object or \[dq]none\[dq]. 39778 39779 By default \[dq]simplejson\[dq]. 39780 Metadata is a small JSON file named after the composite file. 39781 39782 Properties: 39783 39784 - Config: meta_format 39785 - Env Var: RCLONE_CHUNKER_META_FORMAT 39786 - Type: string 39787 - Default: \[dq]simplejson\[dq] 39788 - Examples: 39789 - \[dq]none\[dq] 39790 - Do not use metadata files at all. 39791 - Requires hash type \[dq]none\[dq]. 39792 - \[dq]simplejson\[dq] 39793 - Simple JSON supports hash sums and chunk validation. 39794 - 39795 - It has the following fields: ver, size, nchunks, md5, sha1. 39796 39797 #### --chunker-fail-hard 39798 39799 Choose how chunker should handle files with missing or invalid chunks. 39800 39801 Properties: 39802 39803 - Config: fail_hard 39804 - Env Var: RCLONE_CHUNKER_FAIL_HARD 39805 - Type: bool 39806 - Default: false 39807 - Examples: 39808 - \[dq]true\[dq] 39809 - Report errors and abort current command. 39810 - \[dq]false\[dq] 39811 - Warn user, skip incomplete file and proceed. 39812 39813 #### --chunker-transactions 39814 39815 Choose how chunker should handle temporary files during transactions. 39816 39817 Properties: 39818 39819 - Config: transactions 39820 - Env Var: RCLONE_CHUNKER_TRANSACTIONS 39821 - Type: string 39822 - Default: \[dq]rename\[dq] 39823 - Examples: 39824 - \[dq]rename\[dq] 39825 - Rename temporary files after a successful transaction. 39826 - \[dq]norename\[dq] 39827 - Leave temporary file names and write transaction ID to metadata file. 39828 - Metadata is required for no rename transactions (meta format cannot be \[dq]none\[dq]). 39829 - If you are using norename transactions you should be careful not to downgrade Rclone 39830 - as older versions of Rclone don\[aq]t support this transaction style and will misinterpret 39831 - files manipulated by norename transactions. 39832 - This method is EXPERIMENTAL, don\[aq]t use on production systems. 39833 - \[dq]auto\[dq] 39834 - Rename or norename will be used depending on capabilities of the backend. 39835 - If meta format is set to \[dq]none\[dq], rename transactions will always be used. 39836 - This method is EXPERIMENTAL, don\[aq]t use on production systems. 39837 39838 #### --chunker-description 39839 39840 Description of the remote 39841 39842 Properties: 39843 39844 - Config: description 39845 - Env Var: RCLONE_CHUNKER_DESCRIPTION 39846 - Type: string 39847 - Required: false 39848 39849 39850 39851 # Citrix ShareFile 39852 39853 [Citrix ShareFile](https://sharefile.com) is a secure file sharing and transfer service aimed as business. 39854 39855 ## Configuration 39856 39857 The initial setup for Citrix ShareFile involves getting a token from 39858 Citrix ShareFile which you can in your browser. \[ga]rclone config\[ga] walks you 39859 through it. 39860 39861 Here is an example of how to make a remote called \[ga]remote\[ga]. First run: 39862 39863 rclone config 39864 39865 This will guide you through an interactive setup process: 39866 \f[R] 39867 .fi 39868 .PP 39869 No remotes found, make a new one? 39870 n) New remote s) Set configuration password q) Quit config n/s/q> n 39871 name> remote Type of storage to configure. 39872 Enter a string value. 39873 Press Enter for the default (\[dq]\[dq]). 39874 Choose a number from below, or type in your own value XX / Citrix 39875 Sharefile \ \[dq]sharefile\[dq] Storage> sharefile ** See help for 39876 sharefile backend at: https://rclone.org/sharefile/ ** 39877 .PP 39878 ID of the root folder 39879 .PP 39880 Leave blank to access \[dq]Personal Folders\[dq]. 39881 You can use one of the standard values here or any folder ID (long hex 39882 number ID). 39883 Enter a string value. 39884 Press Enter for the default (\[dq]\[dq]). 39885 Choose a number from below, or type in your own value 1 / Access the 39886 Personal Folders. 39887 (Default) \ \[dq]\[dq] 2 / Access the Favorites folder. 39888 \ \[dq]favorites\[dq] 3 / Access all the shared folders. 39889 \ \[dq]allshared\[dq] 4 / Access all the individual connectors. 39890 \ \[dq]connectors\[dq] 5 / Access the home, favorites, and shared 39891 folders as well as the connectors. 39892 \ \[dq]top\[dq] root_folder_id> Edit advanced config? 39893 (y/n) y) Yes n) No y/n> n Remote config Use web browser to automatically 39894 authenticate rclone with remote? 39895 * Say Y if the machine running rclone has a web browser you can use * 39896 Say N if running rclone on a (remote) machine without web browser access 39897 If not sure try Y. 39898 If Y failed, try N. 39899 y) Yes n) No y/n> y If your browser doesn\[aq]t open automatically go to 39900 the following link: http://127.0.0.1:53682/auth?state=XXX Log in and 39901 authorize rclone for access Waiting for code... 39902 Got code -------------------- [remote] type = sharefile endpoint = 39903 https://XXX.sharefile.com token = 39904 {\[dq]access_token\[dq]:\[dq]XXX\[dq],\[dq]token_type\[dq]:\[dq]bearer\[dq],\[dq]refresh_token\[dq]:\[dq]XXX\[dq],\[dq]expiry\[dq]:\[dq]2019-09-30T19:41:45.878561877+01:00\[dq]} 39905 -------------------- y) Yes this is OK e) Edit this remote d) Delete 39906 this remote y/e/d> y 39907 .IP 39908 .nf 39909 \f[C] 39910 See the [remote setup docs](https://rclone.org/remote_setup/) for how to set it up on a 39911 machine with no Internet browser available. 39912 39913 Note that rclone runs a webserver on your local machine to collect the 39914 token as returned from Citrix ShareFile. This only runs from the moment it opens 39915 your browser to the moment you get back the verification code. This 39916 is on \[ga]http://127.0.0.1:53682/\[ga] and this it may require you to unblock 39917 it temporarily if you are running a host firewall. 39918 39919 Once configured you can then use \[ga]rclone\[ga] like this, 39920 39921 List directories in top level of your ShareFile 39922 39923 rclone lsd remote: 39924 39925 List all the files in your ShareFile 39926 39927 rclone ls remote: 39928 39929 To copy a local directory to an ShareFile directory called backup 39930 39931 rclone copy /home/source remote:backup 39932 39933 Paths may be as deep as required, e.g. \[ga]remote:directory/subdirectory\[ga]. 39934 39935 ### Modification times and hashes 39936 39937 ShareFile allows modification times to be set on objects accurate to 1 39938 second. These will be used to detect whether objects need syncing or 39939 not. 39940 39941 ShareFile supports MD5 type hashes, so you can use the \[ga]--checksum\[ga] 39942 flag. 39943 39944 ### Transfers 39945 39946 For files above 128 MiB rclone will use a chunked transfer. Rclone will 39947 upload up to \[ga]--transfers\[ga] chunks at the same time (shared among all 39948 the multipart uploads). Chunks are buffered in memory and are 39949 normally 64 MiB so increasing \[ga]--transfers\[ga] will increase memory use. 39950 39951 ### Restricted filename characters 39952 39953 In addition to the [default restricted characters set](https://rclone.org/overview/#restricted-characters) 39954 the following characters are also replaced: 39955 39956 | Character | Value | Replacement | 39957 | --------- |:-----:|:-----------:| 39958 | \[rs]\[rs] | 0x5C | \[uFF3C] | 39959 | * | 0x2A | \[uFF0A] | 39960 | < | 0x3C | \[uFF1C] | 39961 | > | 0x3E | \[uFF1E] | 39962 | ? | 0x3F | \[uFF1F] | 39963 | : | 0x3A | \[uFF1A] | 39964 | \[rs]| | 0x7C | \[uFF5C] | 39965 | \[dq] | 0x22 | \[uFF02] | 39966 39967 File names can also not start or end with the following characters. 39968 These only get replaced if they are the first or last character in the 39969 name: 39970 39971 | Character | Value | Replacement | 39972 | --------- |:-----:|:-----------:| 39973 | SP | 0x20 | \[u2420] | 39974 | . | 0x2E | \[uFF0E] | 39975 39976 Invalid UTF-8 bytes will also be [replaced](https://rclone.org/overview/#invalid-utf8), 39977 as they can\[aq]t be used in JSON strings. 39978 39979 39980 ### Standard options 39981 39982 Here are the Standard options specific to sharefile (Citrix Sharefile). 39983 39984 #### --sharefile-client-id 39985 39986 OAuth Client Id. 39987 39988 Leave blank normally. 39989 39990 Properties: 39991 39992 - Config: client_id 39993 - Env Var: RCLONE_SHAREFILE_CLIENT_ID 39994 - Type: string 39995 - Required: false 39996 39997 #### --sharefile-client-secret 39998 39999 OAuth Client Secret. 40000 40001 Leave blank normally. 40002 40003 Properties: 40004 40005 - Config: client_secret 40006 - Env Var: RCLONE_SHAREFILE_CLIENT_SECRET 40007 - Type: string 40008 - Required: false 40009 40010 #### --sharefile-root-folder-id 40011 40012 ID of the root folder. 40013 40014 Leave blank to access \[dq]Personal Folders\[dq]. You can use one of the 40015 standard values here or any folder ID (long hex number ID). 40016 40017 Properties: 40018 40019 - Config: root_folder_id 40020 - Env Var: RCLONE_SHAREFILE_ROOT_FOLDER_ID 40021 - Type: string 40022 - Required: false 40023 - Examples: 40024 - \[dq]\[dq] 40025 - Access the Personal Folders (default). 40026 - \[dq]favorites\[dq] 40027 - Access the Favorites folder. 40028 - \[dq]allshared\[dq] 40029 - Access all the shared folders. 40030 - \[dq]connectors\[dq] 40031 - Access all the individual connectors. 40032 - \[dq]top\[dq] 40033 - Access the home, favorites, and shared folders as well as the connectors. 40034 40035 ### Advanced options 40036 40037 Here are the Advanced options specific to sharefile (Citrix Sharefile). 40038 40039 #### --sharefile-token 40040 40041 OAuth Access Token as a JSON blob. 40042 40043 Properties: 40044 40045 - Config: token 40046 - Env Var: RCLONE_SHAREFILE_TOKEN 40047 - Type: string 40048 - Required: false 40049 40050 #### --sharefile-auth-url 40051 40052 Auth server URL. 40053 40054 Leave blank to use the provider defaults. 40055 40056 Properties: 40057 40058 - Config: auth_url 40059 - Env Var: RCLONE_SHAREFILE_AUTH_URL 40060 - Type: string 40061 - Required: false 40062 40063 #### --sharefile-token-url 40064 40065 Token server url. 40066 40067 Leave blank to use the provider defaults. 40068 40069 Properties: 40070 40071 - Config: token_url 40072 - Env Var: RCLONE_SHAREFILE_TOKEN_URL 40073 - Type: string 40074 - Required: false 40075 40076 #### --sharefile-upload-cutoff 40077 40078 Cutoff for switching to multipart upload. 40079 40080 Properties: 40081 40082 - Config: upload_cutoff 40083 - Env Var: RCLONE_SHAREFILE_UPLOAD_CUTOFF 40084 - Type: SizeSuffix 40085 - Default: 128Mi 40086 40087 #### --sharefile-chunk-size 40088 40089 Upload chunk size. 40090 40091 Must a power of 2 >= 256k. 40092 40093 Making this larger will improve performance, but note that each chunk 40094 is buffered in memory one per transfer. 40095 40096 Reducing this will reduce memory usage but decrease performance. 40097 40098 Properties: 40099 40100 - Config: chunk_size 40101 - Env Var: RCLONE_SHAREFILE_CHUNK_SIZE 40102 - Type: SizeSuffix 40103 - Default: 64Mi 40104 40105 #### --sharefile-endpoint 40106 40107 Endpoint for API calls. 40108 40109 This is usually auto discovered as part of the oauth process, but can 40110 be set manually to something like: https://XXX.sharefile.com 40111 40112 40113 Properties: 40114 40115 - Config: endpoint 40116 - Env Var: RCLONE_SHAREFILE_ENDPOINT 40117 - Type: string 40118 - Required: false 40119 40120 #### --sharefile-encoding 40121 40122 The encoding for the backend. 40123 40124 See the [encoding section in the overview](https://rclone.org/overview/#encoding) for more info. 40125 40126 Properties: 40127 40128 - Config: encoding 40129 - Env Var: RCLONE_SHAREFILE_ENCODING 40130 - Type: Encoding 40131 - Default: Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Ctl,LeftSpace,LeftPeriod,RightSpace,RightPeriod,InvalidUtf8,Dot 40132 40133 #### --sharefile-description 40134 40135 Description of the remote 40136 40137 Properties: 40138 40139 - Config: description 40140 - Env Var: RCLONE_SHAREFILE_DESCRIPTION 40141 - Type: string 40142 - Required: false 40143 40144 40145 ## Limitations 40146 40147 Note that ShareFile is case insensitive so you can\[aq]t have a file called 40148 \[dq]Hello.doc\[dq] and one called \[dq]hello.doc\[dq]. 40149 40150 ShareFile only supports filenames up to 256 characters in length. 40151 40152 \[ga]rclone about\[ga] is not supported by the Citrix ShareFile backend. Backends without 40153 this capability cannot determine free space for an rclone mount or 40154 use policy \[ga]mfs\[ga] (most free space) as a member of an rclone union 40155 remote. 40156 40157 See [List of backends that do not support rclone about](https://rclone.org/overview/#optional-features) and [rclone about](https://rclone.org/commands/rclone_about/) 40158 40159 # Crypt 40160 40161 Rclone \[ga]crypt\[ga] remotes encrypt and decrypt other remotes. 40162 40163 A remote of type \[ga]crypt\[ga] does not access a [storage system](https://rclone.org/overview/) 40164 directly, but instead wraps another remote, which in turn accesses 40165 the storage system. This is similar to how [alias](https://rclone.org/alias/), 40166 [union](https://rclone.org/union/), [chunker](https://rclone.org/chunker/) 40167 and a few others work. It makes the usage very flexible, as you can 40168 add a layer, in this case an encryption layer, on top of any other 40169 backend, even in multiple layers. Rclone\[aq]s functionality 40170 can be used as with any other remote, for example you can 40171 [mount](https://rclone.org/commands/rclone_mount/) a crypt remote. 40172 40173 Accessing a storage system through a crypt remote realizes client-side 40174 encryption, which makes it safe to keep your data in a location you do 40175 not trust will not get compromised. 40176 When working against the \[ga]crypt\[ga] remote, rclone will automatically 40177 encrypt (before uploading) and decrypt (after downloading) on your local 40178 system as needed on the fly, leaving the data encrypted at rest in the 40179 wrapped remote. If you access the storage system using an application 40180 other than rclone, or access the wrapped remote directly using rclone, 40181 there will not be any encryption/decryption: Downloading existing content 40182 will just give you the encrypted (scrambled) format, and anything you 40183 upload will *not* become encrypted. 40184 40185 The encryption is a secret-key encryption (also called symmetric key encryption) 40186 algorithm, where a password (or pass phrase) is used to generate real encryption key. 40187 The password can be supplied by user, or you may chose to let rclone 40188 generate one. It will be stored in the configuration file, in a lightly obscured form. 40189 If you are in an environment where you are not able to keep your configuration 40190 secured, you should add 40191 [configuration encryption](https://rclone.org/docs/#configuration-encryption) 40192 as protection. As long as you have this configuration file, you will be able to 40193 decrypt your data. Without the configuration file, as long as you remember 40194 the password (or keep it in a safe place), you can re-create the configuration 40195 and gain access to the existing data. You may also configure a corresponding 40196 remote in a different installation to access the same data. 40197 See below for guidance to [changing password](#changing-password). 40198 40199 Encryption uses [cryptographic salt](https://en.wikipedia.org/wiki/Salt_(cryptography)), 40200 to permute the encryption key so that the same string may be encrypted in 40201 different ways. When configuring the crypt remote it is optional to enter a salt, 40202 or to let rclone generate a unique salt. If omitted, rclone uses a built-in unique string. 40203 Normally in cryptography, the salt is stored together with the encrypted content, 40204 and do not have to be memorized by the user. This is not the case in rclone, 40205 because rclone does not store any additional information on the remotes. Use of 40206 custom salt is effectively a second password that must be memorized. 40207 40208 [File content](#file-encryption) encryption is performed using 40209 [NaCl SecretBox](https://godoc.org/golang.org/x/crypto/nacl/secretbox), 40210 based on XSalsa20 cipher and Poly1305 for integrity. 40211 [Names](#name-encryption) (file- and directory names) are also encrypted 40212 by default, but this has some implications and is therefore 40213 possible to be turned off. 40214 40215 ## Configuration 40216 40217 Here is an example of how to make a remote called \[ga]secret\[ga]. 40218 40219 To use \[ga]crypt\[ga], first set up the underlying remote. Follow the 40220 \[ga]rclone config\[ga] instructions for the specific backend. 40221 40222 Before configuring the crypt remote, check the underlying remote is 40223 working. In this example the underlying remote is called \[ga]remote\[ga]. 40224 We will configure a path \[ga]path\[ga] within this remote to contain the 40225 encrypted content. Anything inside \[ga]remote:path\[ga] will be encrypted 40226 and anything outside will not. 40227 40228 Configure \[ga]crypt\[ga] using \[ga]rclone config\[ga]. In this example the \[ga]crypt\[ga] 40229 remote is called \[ga]secret\[ga], to differentiate it from the underlying 40230 \[ga]remote\[ga]. 40231 40232 When you are done you can use the crypt remote named \[ga]secret\[ga] just 40233 as you would with any other remote, e.g. \[ga]rclone copy D:\[rs]docs secret:\[rs]docs\[ga], 40234 and rclone will encrypt and decrypt as needed on the fly. 40235 If you access the wrapped remote \[ga]remote:path\[ga] directly you will bypass 40236 the encryption, and anything you read will be in encrypted form, and 40237 anything you write will be unencrypted. To avoid issues it is best to 40238 configure a dedicated path for encrypted content, and access it 40239 exclusively through a crypt remote. 40240 \f[R] 40241 .fi 40242 .PP 40243 No remotes found, make a new one? 40244 n) New remote s) Set configuration password q) Quit config n/s/q> n 40245 name> secret Type of storage to configure. 40246 Enter a string value. 40247 Press Enter for the default (\[dq]\[dq]). 40248 Choose a number from below, or type in your own value [snip] XX / 40249 Encrypt/Decrypt a remote \ \[dq]crypt\[dq] [snip] Storage> crypt ** See 40250 help for crypt backend at: https://rclone.org/crypt/ ** 40251 .PP 40252 Remote to encrypt/decrypt. 40253 Normally should contain a \[aq]:\[aq] and a path, eg 40254 \[dq]myremote:path/to/dir\[dq], \[dq]myremote:bucket\[dq] or maybe 40255 \[dq]myremote:\[dq] (not recommended). 40256 Enter a string value. 40257 Press Enter for the default (\[dq]\[dq]). 40258 remote> remote:path How to encrypt the filenames. 40259 Enter a string value. 40260 Press Enter for the default (\[dq]standard\[dq]). 40261 Choose a number from below, or type in your own value. 40262 / Encrypt the filenames. 40263 1 | See the docs for the details. 40264 \ \[dq]standard\[dq] 2 / Very simple filename obfuscation. 40265 \ \[dq]obfuscate\[dq] / Don\[aq]t encrypt the file names. 40266 3 | Adds a \[dq].bin\[dq] extension only. 40267 \ \[dq]off\[dq] filename_encryption> Option to either encrypt directory 40268 names or leave them intact. 40269 .PP 40270 NB If filename_encryption is \[dq]off\[dq] then this option will do 40271 nothing. 40272 Enter a boolean value (true or false). 40273 Press Enter for the default (\[dq]true\[dq]). 40274 Choose a number from below, or type in your own value 1 / Encrypt 40275 directory names. 40276 \ \[dq]true\[dq] 2 / Don\[aq]t encrypt directory names, leave them 40277 intact. 40278 \ \[dq]false\[dq] directory_name_encryption> Password or pass phrase for 40279 encryption. 40280 y) Yes type in my own password g) Generate random password y/g> y Enter 40281 the password: password: Confirm the password: password: Password or pass 40282 phrase for salt. 40283 Optional but recommended. 40284 Should be different to the previous password. 40285 y) Yes type in my own password g) Generate random password n) No leave 40286 this optional password blank (default) y/g/n> g Password strength in 40287 bits. 40288 64 is just about memorable 128 is secure 1024 is the maximum Bits> 128 40289 Your password is: JAsJvRcgR-_veXNfy_sGmQ Use this password? 40290 Please note that an obscured version of this password (and not the 40291 password itself) will be stored under your configuration file, so keep 40292 this generated password in a safe place. 40293 y) Yes (default) n) No y/n> Edit advanced config? 40294 (y/n) y) Yes n) No (default) y/n> Remote config -------------------- 40295 [secret] type = crypt remote = remote:path password = *** ENCRYPTED 40296 \f[B]\f[BI] password2 = \f[B]\f[R] ENCRYPTED *** -------------------- y) 40297 Yes this is OK (default) e) Edit this remote d) Delete this remote 40298 y/e/d> 40299 .IP 40300 .nf 40301 \f[C] 40302 **Important** The crypt password stored in \[ga]rclone.conf\[ga] is lightly 40303 obscured. That only protects it from cursory inspection. It is not 40304 secure unless [configuration encryption](https://rclone.org/docs/#configuration-encryption) of \[ga]rclone.conf\[ga] is specified. 40305 40306 A long passphrase is recommended, or \[ga]rclone config\[ga] can generate a 40307 random one. 40308 40309 The obscured password is created using AES-CTR with a static key. The 40310 salt is stored verbatim at the beginning of the obscured password. This 40311 static key is shared between all versions of rclone. 40312 40313 If you reconfigure rclone with the same passwords/passphrases 40314 elsewhere it will be compatible, but the obscured version will be different 40315 due to the different salt. 40316 40317 Rclone does not encrypt 40318 40319 * file length - this can be calculated within 16 bytes 40320 * modification time - used for syncing 40321 40322 ### Specifying the remote 40323 40324 When configuring the remote to encrypt/decrypt, you may specify any 40325 string that rclone accepts as a source/destination of other commands. 40326 40327 The primary use case is to specify the path into an already configured 40328 remote (e.g. \[ga]remote:path/to/dir\[ga] or \[ga]remote:bucket\[ga]), such that 40329 data in a remote untrusted location can be stored encrypted. 40330 40331 You may also specify a local filesystem path, such as 40332 \[ga]/path/to/dir\[ga] on Linux, \[ga]C:\[rs]path\[rs]to\[rs]dir\[ga] on Windows. By creating 40333 a crypt remote pointing to such a local filesystem path, you can 40334 use rclone as a utility for pure local file encryption, for example 40335 to keep encrypted files on a removable USB drive. 40336 40337 **Note**: A string which do not contain a \[ga]:\[ga] will by rclone be treated 40338 as a relative path in the local filesystem. For example, if you enter 40339 the name \[ga]remote\[ga] without the trailing \[ga]:\[ga], it will be treated as 40340 a subdirectory of the current directory with name \[dq]remote\[dq]. 40341 40342 If a path \[ga]remote:path/to/dir\[ga] is specified, rclone stores encrypted 40343 files in \[ga]path/to/dir\[ga] on the remote. With file name encryption, files 40344 saved to \[ga]secret:subdir/subfile\[ga] are stored in the unencrypted path 40345 \[ga]path/to/dir\[ga] but the \[ga]subdir/subpath\[ga] element is encrypted. 40346 40347 The path you specify does not have to exist, rclone will create 40348 it when needed. 40349 40350 If you intend to use the wrapped remote both directly for keeping 40351 unencrypted content, as well as through a crypt remote for encrypted 40352 content, it is recommended to point the crypt remote to a separate 40353 directory within the wrapped remote. If you use a bucket-based storage 40354 system (e.g. Swift, S3, Google Compute Storage, B2) it is generally 40355 advisable to wrap the crypt remote around a specific bucket (\[ga]s3:bucket\[ga]). 40356 If wrapping around the entire root of the storage (\[ga]s3:\[ga]), and use the 40357 optional file name encryption, rclone will encrypt the bucket name. 40358 40359 ### Changing password 40360 40361 Should the password, or the configuration file containing a lightly obscured 40362 form of the password, be compromised, you need to re-encrypt your data with 40363 a new password. Since rclone uses secret-key encryption, where the encryption 40364 key is generated directly from the password kept on the client, it is not 40365 possible to change the password/key of already encrypted content. Just changing 40366 the password configured for an existing crypt remote means you will no longer 40367 able to decrypt any of the previously encrypted content. The only possibility 40368 is to re-upload everything via a crypt remote configured with your new password. 40369 40370 Depending on the size of your data, your bandwidth, storage quota etc, there are 40371 different approaches you can take: 40372 - If you have everything in a different location, for example on your local system, 40373 you could remove all of the prior encrypted files, change the password for your 40374 configured crypt remote (or delete and re-create the crypt configuration), 40375 and then re-upload everything from the alternative location. 40376 - If you have enough space on the storage system you can create a new crypt 40377 remote pointing to a separate directory on the same backend, and then use 40378 rclone to copy everything from the original crypt remote to the new, 40379 effectively decrypting everything on the fly using the old password and 40380 re-encrypting using the new password. When done, delete the original crypt 40381 remote directory and finally the rclone crypt configuration with the old password. 40382 All data will be streamed from the storage system and back, so you will 40383 get half the bandwidth and be charged twice if you have upload and download quota 40384 on the storage system. 40385 40386 **Note**: A security problem related to the random password generator 40387 was fixed in rclone version 1.53.3 (released 2020-11-19). Passwords generated 40388 by rclone config in version 1.49.0 (released 2019-08-26) to 1.53.2 40389 (released 2020-10-26) are not considered secure and should be changed. 40390 If you made up your own password, or used rclone version older than 1.49.0 or 40391 newer than 1.53.2 to generate it, you are *not* affected by this issue. 40392 See [issue #4783](https://github.com/artpar/artpar/issues/4783) for more 40393 details, and a tool you can use to check if you are affected. 40394 40395 ### Example 40396 40397 Create the following file structure using \[dq]standard\[dq] file name 40398 encryption. 40399 \f[R] 40400 .fi 40401 .PP 40402 plaintext/ \[u251C]\[u2500]\[u2500] file0.txt \[u251C]\[u2500]\[u2500] 40403 file1.txt \[u2514]\[u2500]\[u2500] subdir \[u251C]\[u2500]\[u2500] 40404 file2.txt \[u251C]\[u2500]\[u2500] file3.txt \[u2514]\[u2500]\[u2500] 40405 subsubdir \[u2514]\[u2500]\[u2500] file4.txt 40406 .IP 40407 .nf 40408 \f[C] 40409 Copy these to the remote, and list them 40410 \f[R] 40411 .fi 40412 .PP 40413 $ rclone -q copy plaintext secret: $ rclone -q ls secret: 7 file1.txt 6 40414 file0.txt 8 subdir/file2.txt 10 subdir/subsubdir/file4.txt 9 40415 subdir/file3.txt 40416 .IP 40417 .nf 40418 \f[C] 40419 The crypt remote looks like 40420 \f[R] 40421 .fi 40422 .PP 40423 $ rclone -q ls remote:path 55 hagjclgavj2mbiqm6u6cnjjqcg 54 40424 v05749mltvv1tf4onltun46gls 57 40425 86vhrsv86mpbtd3a0akjuqslj8/dlj7fkq4kdq72emafg7a7s41uo 58 40426 86vhrsv86mpbtd3a0akjuqslj8/7uu829995du6o42n32otfhjqp4/b9pausrfansjth5ob3jkdqd4lc 40427 56 86vhrsv86mpbtd3a0akjuqslj8/8njh1sk437gttmep3p70g81aps 40428 .IP 40429 .nf 40430 \f[C] 40431 The directory structure is preserved 40432 \f[R] 40433 .fi 40434 .PP 40435 $ rclone -q ls secret:subdir 8 file2.txt 9 file3.txt 10 40436 subsubdir/file4.txt 40437 .IP 40438 .nf 40439 \f[C] 40440 Without file name encryption \[ga].bin\[ga] extensions are added to underlying 40441 names. This prevents the cloud provider attempting to interpret file 40442 content. 40443 \f[R] 40444 .fi 40445 .PP 40446 $ rclone -q ls remote:path 54 file0.txt.bin 57 subdir/file3.txt.bin 56 40447 subdir/file2.txt.bin 58 subdir/subsubdir/file4.txt.bin 55 file1.txt.bin 40448 .IP 40449 .nf 40450 \f[C] 40451 ### File name encryption modes 40452 40453 Off 40454 40455 * doesn\[aq]t hide file names or directory structure 40456 * allows for longer file names (\[ti]246 characters) 40457 * can use sub paths and copy single files 40458 40459 Standard 40460 40461 * file names encrypted 40462 * file names can\[aq]t be as long (\[ti]143 characters) 40463 * can use sub paths and copy single files 40464 * directory structure visible 40465 * identical files names will have identical uploaded names 40466 * can use shortcuts to shorten the directory recursion 40467 40468 Obfuscation 40469 40470 This is a simple \[dq]rotate\[dq] of the filename, with each file having a rot 40471 distance based on the filename. Rclone stores the distance at the 40472 beginning of the filename. A file called \[dq]hello\[dq] may become \[dq]53.jgnnq\[dq]. 40473 40474 Obfuscation is not a strong encryption of filenames, but hinders 40475 automated scanning tools picking up on filename patterns. It is an 40476 intermediate between \[dq]off\[dq] and \[dq]standard\[dq] which allows for longer path 40477 segment names. 40478 40479 There is a possibility with some unicode based filenames that the 40480 obfuscation is weak and may map lower case characters to upper case 40481 equivalents. 40482 40483 Obfuscation cannot be relied upon for strong protection. 40484 40485 * file names very lightly obfuscated 40486 * file names can be longer than standard encryption 40487 * can use sub paths and copy single files 40488 * directory structure visible 40489 * identical files names will have identical uploaded names 40490 40491 Cloud storage systems have limits on file name length and 40492 total path length which rclone is more likely to breach using 40493 \[dq]Standard\[dq] file name encryption. Where file names are less than 156 40494 characters in length issues should not be encountered, irrespective of 40495 cloud storage provider. 40496 40497 An experimental advanced option \[ga]filename_encoding\[ga] is now provided to 40498 address this problem to a certain degree. 40499 For cloud storage systems with case sensitive file names (e.g. Google Drive), 40500 \[ga]base64\[ga] can be used to reduce file name length. 40501 For cloud storage systems using UTF-16 to store file names internally 40502 (e.g. OneDrive, Dropbox, Box), \[ga]base32768\[ga] can be used to drastically reduce 40503 file name length. 40504 40505 An alternative, future rclone file name encryption mode may tolerate 40506 backend provider path length limits. 40507 40508 ### Directory name encryption 40509 40510 Crypt offers the option of encrypting dir names or leaving them intact. 40511 There are two options: 40512 40513 True 40514 40515 Encrypts the whole file path including directory names 40516 Example: 40517 \[ga]1/12/123.txt\[ga] is encrypted to 40518 \[ga]p0e52nreeaj0a5ea7s64m4j72s/l42g6771hnv3an9cgc8cr2n1ng/qgm4avr35m5loi1th53ato71v0\[ga] 40519 40520 False 40521 40522 Only encrypts file names, skips directory names 40523 Example: 40524 \[ga]1/12/123.txt\[ga] is encrypted to 40525 \[ga]1/12/qgm4avr35m5loi1th53ato71v0\[ga] 40526 40527 40528 ### Modification times and hashes 40529 40530 Crypt stores modification times using the underlying remote so support 40531 depends on that. 40532 40533 Hashes are not stored for crypt. However the data integrity is 40534 protected by an extremely strong crypto authenticator. 40535 40536 Use the \[ga]rclone cryptcheck\[ga] command to check the 40537 integrity of an encrypted remote instead of \[ga]rclone check\[ga] which can\[aq]t 40538 check the checksums properly. 40539 40540 40541 ### Standard options 40542 40543 Here are the Standard options specific to crypt (Encrypt/Decrypt a remote). 40544 40545 #### --crypt-remote 40546 40547 Remote to encrypt/decrypt. 40548 40549 Normally should contain a \[aq]:\[aq] and a path, e.g. \[dq]myremote:path/to/dir\[dq], 40550 \[dq]myremote:bucket\[dq] or maybe \[dq]myremote:\[dq] (not recommended). 40551 40552 Properties: 40553 40554 - Config: remote 40555 - Env Var: RCLONE_CRYPT_REMOTE 40556 - Type: string 40557 - Required: true 40558 40559 #### --crypt-filename-encryption 40560 40561 How to encrypt the filenames. 40562 40563 Properties: 40564 40565 - Config: filename_encryption 40566 - Env Var: RCLONE_CRYPT_FILENAME_ENCRYPTION 40567 - Type: string 40568 - Default: \[dq]standard\[dq] 40569 - Examples: 40570 - \[dq]standard\[dq] 40571 - Encrypt the filenames. 40572 - See the docs for the details. 40573 - \[dq]obfuscate\[dq] 40574 - Very simple filename obfuscation. 40575 - \[dq]off\[dq] 40576 - Don\[aq]t encrypt the file names. 40577 - Adds a \[dq].bin\[dq], or \[dq]suffix\[dq] extension only. 40578 40579 #### --crypt-directory-name-encryption 40580 40581 Option to either encrypt directory names or leave them intact. 40582 40583 NB If filename_encryption is \[dq]off\[dq] then this option will do nothing. 40584 40585 Properties: 40586 40587 - Config: directory_name_encryption 40588 - Env Var: RCLONE_CRYPT_DIRECTORY_NAME_ENCRYPTION 40589 - Type: bool 40590 - Default: true 40591 - Examples: 40592 - \[dq]true\[dq] 40593 - Encrypt directory names. 40594 - \[dq]false\[dq] 40595 - Don\[aq]t encrypt directory names, leave them intact. 40596 40597 #### --crypt-password 40598 40599 Password or pass phrase for encryption. 40600 40601 **NB** Input to this must be obscured - see [rclone obscure](https://rclone.org/commands/rclone_obscure/). 40602 40603 Properties: 40604 40605 - Config: password 40606 - Env Var: RCLONE_CRYPT_PASSWORD 40607 - Type: string 40608 - Required: true 40609 40610 #### --crypt-password2 40611 40612 Password or pass phrase for salt. 40613 40614 Optional but recommended. 40615 Should be different to the previous password. 40616 40617 **NB** Input to this must be obscured - see [rclone obscure](https://rclone.org/commands/rclone_obscure/). 40618 40619 Properties: 40620 40621 - Config: password2 40622 - Env Var: RCLONE_CRYPT_PASSWORD2 40623 - Type: string 40624 - Required: false 40625 40626 ### Advanced options 40627 40628 Here are the Advanced options specific to crypt (Encrypt/Decrypt a remote). 40629 40630 #### --crypt-server-side-across-configs 40631 40632 Deprecated: use --server-side-across-configs instead. 40633 40634 Allow server-side operations (e.g. copy) to work across different crypt configs. 40635 40636 Normally this option is not what you want, but if you have two crypts 40637 pointing to the same backend you can use it. 40638 40639 This can be used, for example, to change file name encryption type 40640 without re-uploading all the data. Just make two crypt backends 40641 pointing to two different directories with the single changed 40642 parameter and use rclone move to move the files between the crypt 40643 remotes. 40644 40645 Properties: 40646 40647 - Config: server_side_across_configs 40648 - Env Var: RCLONE_CRYPT_SERVER_SIDE_ACROSS_CONFIGS 40649 - Type: bool 40650 - Default: false 40651 40652 #### --crypt-show-mapping 40653 40654 For all files listed show how the names encrypt. 40655 40656 If this flag is set then for each file that the remote is asked to 40657 list, it will log (at level INFO) a line stating the decrypted file 40658 name and the encrypted file name. 40659 40660 This is so you can work out which encrypted names are which decrypted 40661 names just in case you need to do something with the encrypted file 40662 names, or for debugging purposes. 40663 40664 Properties: 40665 40666 - Config: show_mapping 40667 - Env Var: RCLONE_CRYPT_SHOW_MAPPING 40668 - Type: bool 40669 - Default: false 40670 40671 #### --crypt-no-data-encryption 40672 40673 Option to either encrypt file data or leave it unencrypted. 40674 40675 Properties: 40676 40677 - Config: no_data_encryption 40678 - Env Var: RCLONE_CRYPT_NO_DATA_ENCRYPTION 40679 - Type: bool 40680 - Default: false 40681 - Examples: 40682 - \[dq]true\[dq] 40683 - Don\[aq]t encrypt file data, leave it unencrypted. 40684 - \[dq]false\[dq] 40685 - Encrypt file data. 40686 40687 #### --crypt-pass-bad-blocks 40688 40689 If set this will pass bad blocks through as all 0. 40690 40691 This should not be set in normal operation, it should only be set if 40692 trying to recover an encrypted file with errors and it is desired to 40693 recover as much of the file as possible. 40694 40695 Properties: 40696 40697 - Config: pass_bad_blocks 40698 - Env Var: RCLONE_CRYPT_PASS_BAD_BLOCKS 40699 - Type: bool 40700 - Default: false 40701 40702 #### --crypt-strict-names 40703 40704 If set, this will raise an error when crypt comes across a filename that can\[aq]t be decrypted. 40705 40706 (By default, rclone will just log a NOTICE and continue as normal.) 40707 This can happen if encrypted and unencrypted files are stored in the same 40708 directory (which is not recommended.) It may also indicate a more serious 40709 problem that should be investigated. 40710 40711 Properties: 40712 40713 - Config: strict_names 40714 - Env Var: RCLONE_CRYPT_STRICT_NAMES 40715 - Type: bool 40716 - Default: false 40717 40718 #### --crypt-filename-encoding 40719 40720 How to encode the encrypted filename to text string. 40721 40722 This option could help with shortening the encrypted filename. The 40723 suitable option would depend on the way your remote count the filename 40724 length and if it\[aq]s case sensitive. 40725 40726 Properties: 40727 40728 - Config: filename_encoding 40729 - Env Var: RCLONE_CRYPT_FILENAME_ENCODING 40730 - Type: string 40731 - Default: \[dq]base32\[dq] 40732 - Examples: 40733 - \[dq]base32\[dq] 40734 - Encode using base32. Suitable for all remote. 40735 - \[dq]base64\[dq] 40736 - Encode using base64. Suitable for case sensitive remote. 40737 - \[dq]base32768\[dq] 40738 - Encode using base32768. Suitable if your remote counts UTF-16 or 40739 - Unicode codepoint instead of UTF-8 byte length. (Eg. Onedrive, Dropbox) 40740 40741 #### --crypt-suffix 40742 40743 If this is set it will override the default suffix of \[dq].bin\[dq]. 40744 40745 Setting suffix to \[dq]none\[dq] will result in an empty suffix. This may be useful 40746 when the path length is critical. 40747 40748 Properties: 40749 40750 - Config: suffix 40751 - Env Var: RCLONE_CRYPT_SUFFIX 40752 - Type: string 40753 - Default: \[dq].bin\[dq] 40754 40755 #### --crypt-description 40756 40757 Description of the remote 40758 40759 Properties: 40760 40761 - Config: description 40762 - Env Var: RCLONE_CRYPT_DESCRIPTION 40763 - Type: string 40764 - Required: false 40765 40766 ### Metadata 40767 40768 Any metadata supported by the underlying remote is read and written. 40769 40770 See the [metadata](https://rclone.org/docs/#metadata) docs for more info. 40771 40772 ## Backend commands 40773 40774 Here are the commands specific to the crypt backend. 40775 40776 Run them with 40777 40778 rclone backend COMMAND remote: 40779 40780 The help below will explain what arguments each command takes. 40781 40782 See the [backend](https://rclone.org/commands/rclone_backend/) command for more 40783 info on how to pass options and arguments. 40784 40785 These can be run on a running backend using the rc command 40786 [backend/command](https://rclone.org/rc/#backend-command). 40787 40788 ### encode 40789 40790 Encode the given filename(s) 40791 40792 rclone backend encode remote: [options] [<arguments>+] 40793 40794 This encodes the filenames given as arguments returning a list of 40795 strings of the encoded results. 40796 40797 Usage Example: 40798 40799 rclone backend encode crypt: file1 [file2...] 40800 rclone rc backend/command command=encode fs=crypt: file1 [file2...] 40801 40802 40803 ### decode 40804 40805 Decode the given filename(s) 40806 40807 rclone backend decode remote: [options] [<arguments>+] 40808 40809 This decodes the filenames given as arguments returning a list of 40810 strings of the decoded results. It will return an error if any of the 40811 inputs are invalid. 40812 40813 Usage Example: 40814 40815 rclone backend decode crypt: encryptedfile1 [encryptedfile2...] 40816 rclone rc backend/command command=decode fs=crypt: encryptedfile1 [encryptedfile2...] 40817 40818 40819 40820 40821 ## Backing up an encrypted remote 40822 40823 If you wish to backup an encrypted remote, it is recommended that you use 40824 \[ga]rclone sync\[ga] on the encrypted files, and make sure the passwords are 40825 the same in the new encrypted remote. 40826 40827 This will have the following advantages 40828 40829 * \[ga]rclone sync\[ga] will check the checksums while copying 40830 * you can use \[ga]rclone check\[ga] between the encrypted remotes 40831 * you don\[aq]t decrypt and encrypt unnecessarily 40832 40833 For example, let\[aq]s say you have your original remote at \[ga]remote:\[ga] with 40834 the encrypted version at \[ga]eremote:\[ga] with path \[ga]remote:crypt\[ga]. You 40835 would then set up the new remote \[ga]remote2:\[ga] and then the encrypted 40836 version \[ga]eremote2:\[ga] with path \[ga]remote2:crypt\[ga] using the same passwords 40837 as \[ga]eremote:\[ga]. 40838 40839 To sync the two remotes you would do 40840 40841 rclone sync --interactive remote:crypt remote2:crypt 40842 40843 And to check the integrity you would do 40844 40845 rclone check remote:crypt remote2:crypt 40846 40847 ## File formats 40848 40849 ### File encryption 40850 40851 Files are encrypted 1:1 source file to destination object. The file 40852 has a header and is divided into chunks. 40853 40854 #### Header 40855 40856 * 8 bytes magic string \[ga]RCLONE\[rs]x00\[rs]x00\[ga] 40857 * 24 bytes Nonce (IV) 40858 40859 The initial nonce is generated from the operating systems crypto 40860 strong random number generator. The nonce is incremented for each 40861 chunk read making sure each nonce is unique for each block written. 40862 The chance of a nonce being reused is minuscule. If you wrote an 40863 exabyte of data (10\[S1]\[u2078] bytes) you would have a probability of 40864 approximately 2\[tmu]10\[u207B]\[S3]\[S2] of re-using a nonce. 40865 40866 #### Chunk 40867 40868 Each chunk will contain 64 KiB of data, except for the last one which 40869 may have less data. The data chunk is in standard NaCl SecretBox 40870 format. SecretBox uses XSalsa20 and Poly1305 to encrypt and 40871 authenticate messages. 40872 40873 Each chunk contains: 40874 40875 * 16 Bytes of Poly1305 authenticator 40876 * 1 - 65536 bytes XSalsa20 encrypted data 40877 40878 64k chunk size was chosen as the best performing chunk size (the 40879 authenticator takes too much time below this and the performance drops 40880 off due to cache effects above this). Note that these chunks are 40881 buffered in memory so they can\[aq]t be too big. 40882 40883 This uses a 32 byte (256 bit key) key derived from the user password. 40884 40885 #### Examples 40886 40887 1 byte file will encrypt to 40888 40889 * 32 bytes header 40890 * 17 bytes data chunk 40891 40892 49 bytes total 40893 40894 1 MiB (1048576 bytes) file will encrypt to 40895 40896 * 32 bytes header 40897 * 16 chunks of 65568 bytes 40898 40899 1049120 bytes total (a 0.05% overhead). This is the overhead for big 40900 files. 40901 40902 ### Name encryption 40903 40904 File names are encrypted segment by segment - the path is broken up 40905 into \[ga]/\[ga] separated strings and these are encrypted individually. 40906 40907 File segments are padded using PKCS#7 to a multiple of 16 bytes 40908 before encryption. 40909 40910 They are then encrypted with EME using AES with 256 bit key. EME 40911 (ECB-Mix-ECB) is a wide-block encryption mode presented in the 2003 40912 paper \[dq]A Parallelizable Enciphering Mode\[dq] by Halevi and Rogaway. 40913 40914 This makes for deterministic encryption which is what we want - the 40915 same filename must encrypt to the same thing otherwise we can\[aq]t find 40916 it on the cloud storage system. 40917 40918 This means that 40919 40920 * filenames with the same name will encrypt the same 40921 * filenames which start the same won\[aq]t have a common prefix 40922 40923 This uses a 32 byte key (256 bits) and a 16 byte (128 bits) IV both of 40924 which are derived from the user password. 40925 40926 After encryption they are written out using a modified version of 40927 standard \[ga]base32\[ga] encoding as described in RFC4648. The standard 40928 encoding is modified in two ways: 40929 40930 * it becomes lower case (no-one likes upper case filenames!) 40931 * we strip the padding character \[ga]=\[ga] 40932 40933 \[ga]base32\[ga] is used rather than the more efficient \[ga]base64\[ga] so rclone can be 40934 used on case insensitive remotes (e.g. Windows, Box, Dropbox, Onedrive etc). 40935 40936 ### Key derivation 40937 40938 Rclone uses \[ga]scrypt\[ga] with parameters \[ga]N=16384, r=8, p=1\[ga] with an 40939 optional user supplied salt (password2) to derive the 32+32+16 = 80 40940 bytes of key material required. If the user doesn\[aq]t supply a salt 40941 then rclone uses an internal one. 40942 40943 \[ga]scrypt\[ga] makes it impractical to mount a dictionary attack on rclone 40944 encrypted data. For full protection against this you should always use 40945 a salt. 40946 40947 ## SEE ALSO 40948 40949 * [rclone cryptdecode](https://rclone.org/commands/rclone_cryptdecode/) - Show forward/reverse mapping of encrypted filenames 40950 40951 # Compress 40952 40953 ## Warning 40954 40955 This remote is currently **experimental**. Things may break and data may be lost. Anything you do with this remote is 40956 at your own risk. Please understand the risks associated with using experimental code and don\[aq]t use this remote in 40957 critical applications. 40958 40959 The \[ga]Compress\[ga] remote adds compression to another remote. It is best used with remotes containing 40960 many large compressible files. 40961 40962 ## Configuration 40963 40964 To use this remote, all you need to do is specify another remote and a compression mode to use: 40965 \f[R] 40966 .fi 40967 .PP 40968 Current remotes: 40969 .PP 40970 Name Type ==== ==== remote_to_press sometype 40971 .IP "e)" 3 40972 Edit existing remote $ rclone config 40973 .IP "f)" 3 40974 New remote 40975 .IP "g)" 3 40976 Delete remote 40977 .IP "h)" 3 40978 Rename remote 40979 .IP "i)" 3 40980 Copy remote 40981 .IP "j)" 3 40982 Set configuration password 40983 .IP "k)" 3 40984 Quit config e/n/d/r/c/s/q> n name> compress ... 40985 8 / Compress a remote \ \[dq]compress\[dq] ... 40986 Storage> compress ** See help for compress backend at: 40987 https://rclone.org/compress/ ** 40988 .PP 40989 Remote to compress. 40990 Enter a string value. 40991 Press Enter for the default (\[dq]\[dq]). 40992 remote> remote_to_press:subdir Compression mode. 40993 Enter a string value. 40994 Press Enter for the default (\[dq]gzip\[dq]). 40995 Choose a number from below, or type in your own value 1 / Gzip 40996 compression balanced for speed and compression strength. 40997 \ \[dq]gzip\[dq] compression_mode> gzip Edit advanced config? 40998 (y/n) y) Yes n) No (default) y/n> n Remote config -------------------- 40999 [compress] type = compress remote = remote_to_press:subdir 41000 compression_mode = gzip -------------------- y) Yes this is OK (default) 41001 e) Edit this remote d) Delete this remote y/e/d> y 41002 .IP 41003 .nf 41004 \f[C] 41005 ### Compression Modes 41006 41007 Currently only gzip compression is supported. It provides a decent balance between speed and size and is well 41008 supported by other applications. Compression strength can further be configured via an advanced setting where 0 is no 41009 compression and 9 is strongest compression. 41010 41011 ### File types 41012 41013 If you open a remote wrapped by compress, you will see that there are many files with an extension corresponding to 41014 the compression algorithm you chose. These files are standard files that can be opened by various archive programs, 41015 but they have some hidden metadata that allows them to be used by rclone. 41016 While you may download and decompress these files at will, do **not** manually delete or rename files. Files without 41017 correct metadata files will not be recognized by rclone. 41018 41019 ### File names 41020 41021 The compressed files will be named \[ga]*.###########.gz\[ga] where \[ga]*\[ga] is the base file and the \[ga]#\[ga] part is base64 encoded 41022 size of the uncompressed file. The file names should not be changed by anything other than the rclone compression backend. 41023 41024 41025 ### Standard options 41026 41027 Here are the Standard options specific to compress (Compress a remote). 41028 41029 #### --compress-remote 41030 41031 Remote to compress. 41032 41033 Properties: 41034 41035 - Config: remote 41036 - Env Var: RCLONE_COMPRESS_REMOTE 41037 - Type: string 41038 - Required: true 41039 41040 #### --compress-mode 41041 41042 Compression mode. 41043 41044 Properties: 41045 41046 - Config: mode 41047 - Env Var: RCLONE_COMPRESS_MODE 41048 - Type: string 41049 - Default: \[dq]gzip\[dq] 41050 - Examples: 41051 - \[dq]gzip\[dq] 41052 - Standard gzip compression with fastest parameters. 41053 41054 ### Advanced options 41055 41056 Here are the Advanced options specific to compress (Compress a remote). 41057 41058 #### --compress-level 41059 41060 GZIP compression level (-2 to 9). 41061 41062 Generally -1 (default, equivalent to 5) is recommended. 41063 Levels 1 to 9 increase compression at the cost of speed. Going past 6 41064 generally offers very little return. 41065 41066 Level -2 uses Huffman encoding only. Only use if you know what you 41067 are doing. 41068 Level 0 turns off compression. 41069 41070 Properties: 41071 41072 - Config: level 41073 - Env Var: RCLONE_COMPRESS_LEVEL 41074 - Type: int 41075 - Default: -1 41076 41077 #### --compress-ram-cache-limit 41078 41079 Some remotes don\[aq]t allow the upload of files with unknown size. 41080 In this case the compressed file will need to be cached to determine 41081 it\[aq]s size. 41082 41083 Files smaller than this limit will be cached in RAM, files larger than 41084 this limit will be cached on disk. 41085 41086 Properties: 41087 41088 - Config: ram_cache_limit 41089 - Env Var: RCLONE_COMPRESS_RAM_CACHE_LIMIT 41090 - Type: SizeSuffix 41091 - Default: 20Mi 41092 41093 #### --compress-description 41094 41095 Description of the remote 41096 41097 Properties: 41098 41099 - Config: description 41100 - Env Var: RCLONE_COMPRESS_DESCRIPTION 41101 - Type: string 41102 - Required: false 41103 41104 ### Metadata 41105 41106 Any metadata supported by the underlying remote is read and written. 41107 41108 See the [metadata](https://rclone.org/docs/#metadata) docs for more info. 41109 41110 41111 41112 # Combine 41113 41114 The \[ga]combine\[ga] backend joins remotes together into a single directory 41115 tree. 41116 41117 For example you might have a remote for images on one provider: 41118 \f[R] 41119 .fi 41120 .PP 41121 $ rclone tree s3:imagesbucket / \[u251C]\[u2500]\[u2500] image1.jpg 41122 \[u2514]\[u2500]\[u2500] image2.jpg 41123 .IP 41124 .nf 41125 \f[C] 41126 And a remote for files on another: 41127 \f[R] 41128 .fi 41129 .PP 41130 $ rclone tree drive:important/files / \[u251C]\[u2500]\[u2500] file1.txt 41131 \[u2514]\[u2500]\[u2500] file2.txt 41132 .IP 41133 .nf 41134 \f[C] 41135 The \[ga]combine\[ga] backend can join these together into a synthetic 41136 directory structure like this: 41137 \f[R] 41138 .fi 41139 .PP 41140 $ rclone tree combined: / \[u251C]\[u2500]\[u2500] files \[br] 41141 \[u251C]\[u2500]\[u2500] file1.txt \[br] \[u2514]\[u2500]\[u2500] 41142 file2.txt \[u2514]\[u2500]\[u2500] images \[u251C]\[u2500]\[u2500] 41143 image1.jpg \[u2514]\[u2500]\[u2500] image2.jpg 41144 .IP 41145 .nf 41146 \f[C] 41147 You\[aq]d do this by specifying an \[ga]upstreams\[ga] parameter in the config 41148 like this 41149 41150 upstreams = images=s3:imagesbucket files=drive:important/files 41151 41152 During the initial setup with \[ga]rclone config\[ga] you will specify the 41153 upstreams remotes as a space separated list. The upstream remotes can 41154 either be a local paths or other remotes. 41155 41156 ## Configuration 41157 41158 Here is an example of how to make a combine called \[ga]remote\[ga] for the 41159 example above. First run: 41160 41161 rclone config 41162 41163 This will guide you through an interactive setup process: 41164 \f[R] 41165 .fi 41166 .PP 41167 No remotes found, make a new one? 41168 n) New remote s) Set configuration password q) Quit config n/s/q> n 41169 name> remote Option Storage. 41170 Type of storage to configure. 41171 Choose a number from below, or type in your own value. 41172 \&... 41173 XX / Combine several remotes into one \ (combine) ... 41174 Storage> combine Option upstreams. 41175 Upstreams for combining These should be in the form dir=remote:path 41176 dir2=remote2:path Where before the = is specified the root directory and 41177 after is the remote to put there. 41178 Embedded spaces can be added using quotes \[dq]dir=remote:path with 41179 space\[dq] \[dq]dir2=remote2:path with space\[dq] Enter a 41180 fs.SpaceSepList value. 41181 upstreams> images=s3:imagesbucket files=drive:important/files 41182 -------------------- [remote] type = combine upstreams = 41183 images=s3:imagesbucket files=drive:important/files -------------------- 41184 y) Yes this is OK (default) e) Edit this remote d) Delete this remote 41185 y/e/d> y 41186 .IP 41187 .nf 41188 \f[C] 41189 ### Configuring for Google Drive Shared Drives 41190 41191 Rclone has a convenience feature for making a combine backend for all 41192 the shared drives you have access to. 41193 41194 Assuming your main (non shared drive) Google drive remote is called 41195 \[ga]drive:\[ga] you would run 41196 41197 rclone backend -o config drives drive: 41198 41199 This would produce something like this: 41200 41201 [My Drive] 41202 type = alias 41203 remote = drive,team_drive=0ABCDEF-01234567890,root_folder_id=: 41204 41205 [Test Drive] 41206 type = alias 41207 remote = drive,team_drive=0ABCDEFabcdefghijkl,root_folder_id=: 41208 41209 [AllDrives] 41210 type = combine 41211 upstreams = \[dq]My Drive=My Drive:\[dq] \[dq]Test Drive=Test Drive:\[dq] 41212 41213 If you then add that config to your config file (find it with \[ga]rclone 41214 config file\[ga]) then you can access all the shared drives in one place 41215 with the \[ga]AllDrives:\[ga] remote. 41216 41217 See [the Google Drive docs](https://rclone.org/drive/#drives) for full info. 41218 41219 41220 ### Standard options 41221 41222 Here are the Standard options specific to combine (Combine several remotes into one). 41223 41224 #### --combine-upstreams 41225 41226 Upstreams for combining 41227 41228 These should be in the form 41229 41230 dir=remote:path dir2=remote2:path 41231 41232 Where before the = is specified the root directory and after is the remote to 41233 put there. 41234 41235 Embedded spaces can be added using quotes 41236 41237 \[dq]dir=remote:path with space\[dq] \[dq]dir2=remote2:path with space\[dq] 41238 41239 41240 41241 Properties: 41242 41243 - Config: upstreams 41244 - Env Var: RCLONE_COMBINE_UPSTREAMS 41245 - Type: SpaceSepList 41246 - Default: 41247 41248 ### Advanced options 41249 41250 Here are the Advanced options specific to combine (Combine several remotes into one). 41251 41252 #### --combine-description 41253 41254 Description of the remote 41255 41256 Properties: 41257 41258 - Config: description 41259 - Env Var: RCLONE_COMBINE_DESCRIPTION 41260 - Type: string 41261 - Required: false 41262 41263 ### Metadata 41264 41265 Any metadata supported by the underlying remote is read and written. 41266 41267 See the [metadata](https://rclone.org/docs/#metadata) docs for more info. 41268 41269 41270 41271 # Dropbox 41272 41273 Paths are specified as \[ga]remote:path\[ga] 41274 41275 Dropbox paths may be as deep as required, e.g. 41276 \[ga]remote:directory/subdirectory\[ga]. 41277 41278 ## Configuration 41279 41280 The initial setup for dropbox involves getting a token from Dropbox 41281 which you need to do in your browser. \[ga]rclone config\[ga] walks you 41282 through it. 41283 41284 Here is an example of how to make a remote called \[ga]remote\[ga]. First run: 41285 41286 rclone config 41287 41288 This will guide you through an interactive setup process: 41289 \f[R] 41290 .fi 41291 .IP "n)" 3 41292 New remote 41293 .IP "o)" 3 41294 Delete remote 41295 .IP "p)" 3 41296 Quit config e/n/d/q> n name> remote Type of storage to configure. 41297 Choose a number from below, or type in your own value [snip] XX / 41298 Dropbox \ \[dq]dropbox\[dq] [snip] Storage> dropbox Dropbox App Key - 41299 leave blank normally. 41300 app_key> Dropbox App Secret - leave blank normally. 41301 app_secret> Remote config Please visit: 41302 https://www.dropbox.com/1/oauth2/authorize?client_id=XXXXXXXXXXXXXXX&response_type=code 41303 Enter the code: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX_XXXXXXXXXX 41304 -------------------- [remote] app_key = app_secret = token = 41305 XXXXXXXXXXXXXXXXXXXXXXXXXXXXX_XXXX_XXXXXXXXXXXXXXXXXXXXXXXXXXXXX 41306 -------------------- 41307 .IP "q)" 3 41308 Yes this is OK 41309 .IP "r)" 3 41310 Edit this remote 41311 .IP "s)" 3 41312 Delete this remote y/e/d> y 41313 .IP 41314 .nf 41315 \f[C] 41316 See the [remote setup docs](https://rclone.org/remote_setup/) for how to set it up on a 41317 machine with no Internet browser available. 41318 41319 Note that rclone runs a webserver on your local machine to collect the 41320 token as returned from Dropbox. This only 41321 runs from the moment it opens your browser to the moment you get back 41322 the verification code. This is on \[ga]http://127.0.0.1:53682/\[ga] and it 41323 may require you to unblock it temporarily if you are running a host 41324 firewall, or use manual mode. 41325 41326 You can then use it like this, 41327 41328 List directories in top level of your dropbox 41329 41330 rclone lsd remote: 41331 41332 List all the files in your dropbox 41333 41334 rclone ls remote: 41335 41336 To copy a local directory to a dropbox directory called backup 41337 41338 rclone copy /home/source remote:backup 41339 41340 ### Dropbox for business 41341 41342 Rclone supports Dropbox for business and Team Folders. 41343 41344 When using Dropbox for business \[ga]remote:\[ga] and \[ga]remote:path/to/file\[ga] 41345 will refer to your personal folder. 41346 41347 If you wish to see Team Folders you must use a leading \[ga]/\[ga] in the 41348 path, so \[ga]rclone lsd remote:/\[ga] will refer to the root and show you all 41349 Team Folders and your User Folder. 41350 41351 You can then use team folders like this \[ga]remote:/TeamFolder\[ga] and 41352 \[ga]remote:/TeamFolder/path/to/file\[ga]. 41353 41354 A leading \[ga]/\[ga] for a Dropbox personal account will do nothing, but it 41355 will take an extra HTTP transaction so it should be avoided. 41356 41357 ### Modification times and hashes 41358 41359 Dropbox supports modified times, but the only way to set a 41360 modification time is to re-upload the file. 41361 41362 This means that if you uploaded your data with an older version of 41363 rclone which didn\[aq]t support the v2 API and modified times, rclone will 41364 decide to upload all your old data to fix the modification times. If 41365 you don\[aq]t want this to happen use \[ga]--size-only\[ga] or \[ga]--checksum\[ga] flag 41366 to stop it. 41367 41368 Dropbox supports [its own hash 41369 type](https://www.dropbox.com/developers/reference/content-hash) which 41370 is checked for all transfers. 41371 41372 ### Restricted filename characters 41373 41374 | Character | Value | Replacement | 41375 | --------- |:-----:|:-----------:| 41376 | NUL | 0x00 | \[u2400] | 41377 | / | 0x2F | \[uFF0F] | 41378 | DEL | 0x7F | \[u2421] | 41379 | \[rs] | 0x5C | \[uFF3C] | 41380 41381 File names can also not end with the following characters. 41382 These only get replaced if they are the last character in the name: 41383 41384 | Character | Value | Replacement | 41385 | --------- |:-----:|:-----------:| 41386 | SP | 0x20 | \[u2420] | 41387 41388 Invalid UTF-8 bytes will also be [replaced](https://rclone.org/overview/#invalid-utf8), 41389 as they can\[aq]t be used in JSON strings. 41390 41391 ### Batch mode uploads {#batch-mode} 41392 41393 Using batch mode uploads is very important for performance when using 41394 the Dropbox API. See [the dropbox performance guide](https://developers.dropbox.com/dbx-performance-guide) 41395 for more info. 41396 41397 There are 3 modes rclone can use for uploads. 41398 41399 #### --dropbox-batch-mode off 41400 41401 In this mode rclone will not use upload batching. This was the default 41402 before rclone v1.55. It has the disadvantage that it is very likely to 41403 encounter \[ga]too_many_requests\[ga] errors like this 41404 41405 NOTICE: too_many_requests/.: Too many requests or write operations. Trying again in 15 seconds. 41406 41407 When rclone receives these it has to wait for 15s or sometimes 300s 41408 before continuing which really slows down transfers. 41409 41410 This will happen especially if \[ga]--transfers\[ga] is large, so this mode 41411 isn\[aq]t recommended except for compatibility or investigating problems. 41412 41413 #### --dropbox-batch-mode sync 41414 41415 In this mode rclone will batch up uploads to the size specified by 41416 \[ga]--dropbox-batch-size\[ga] and commit them together. 41417 41418 Using this mode means you can use a much higher \[ga]--transfers\[ga] 41419 parameter (32 or 64 works fine) without receiving \[ga]too_many_requests\[ga] 41420 errors. 41421 41422 This mode ensures full data integrity. 41423 41424 Note that there may be a pause when quitting rclone while rclone 41425 finishes up the last batch using this mode. 41426 41427 #### --dropbox-batch-mode async 41428 41429 In this mode rclone will batch up uploads to the size specified by 41430 \[ga]--dropbox-batch-size\[ga] and commit them together. 41431 41432 However it will not wait for the status of the batch to be returned to 41433 the caller. This means rclone can use a much bigger batch size (much 41434 bigger than \[ga]--transfers\[ga]), at the cost of not being able to check the 41435 status of the upload. 41436 41437 This provides the maximum possible upload speed especially with lots 41438 of small files, however rclone can\[aq]t check the file got uploaded 41439 properly using this mode. 41440 41441 If you are using this mode then using \[dq]rclone check\[dq] after the 41442 transfer completes is recommended. Or you could do an initial transfer 41443 with \[ga]--dropbox-batch-mode async\[ga] then do a final transfer with 41444 \[ga]--dropbox-batch-mode sync\[ga] (the default). 41445 41446 Note that there may be a pause when quitting rclone while rclone 41447 finishes up the last batch using this mode. 41448 41449 41450 41451 ### Standard options 41452 41453 Here are the Standard options specific to dropbox (Dropbox). 41454 41455 #### --dropbox-client-id 41456 41457 OAuth Client Id. 41458 41459 Leave blank normally. 41460 41461 Properties: 41462 41463 - Config: client_id 41464 - Env Var: RCLONE_DROPBOX_CLIENT_ID 41465 - Type: string 41466 - Required: false 41467 41468 #### --dropbox-client-secret 41469 41470 OAuth Client Secret. 41471 41472 Leave blank normally. 41473 41474 Properties: 41475 41476 - Config: client_secret 41477 - Env Var: RCLONE_DROPBOX_CLIENT_SECRET 41478 - Type: string 41479 - Required: false 41480 41481 ### Advanced options 41482 41483 Here are the Advanced options specific to dropbox (Dropbox). 41484 41485 #### --dropbox-token 41486 41487 OAuth Access Token as a JSON blob. 41488 41489 Properties: 41490 41491 - Config: token 41492 - Env Var: RCLONE_DROPBOX_TOKEN 41493 - Type: string 41494 - Required: false 41495 41496 #### --dropbox-auth-url 41497 41498 Auth server URL. 41499 41500 Leave blank to use the provider defaults. 41501 41502 Properties: 41503 41504 - Config: auth_url 41505 - Env Var: RCLONE_DROPBOX_AUTH_URL 41506 - Type: string 41507 - Required: false 41508 41509 #### --dropbox-token-url 41510 41511 Token server url. 41512 41513 Leave blank to use the provider defaults. 41514 41515 Properties: 41516 41517 - Config: token_url 41518 - Env Var: RCLONE_DROPBOX_TOKEN_URL 41519 - Type: string 41520 - Required: false 41521 41522 #### --dropbox-chunk-size 41523 41524 Upload chunk size (< 150Mi). 41525 41526 Any files larger than this will be uploaded in chunks of this size. 41527 41528 Note that chunks are buffered in memory (one at a time) so rclone can 41529 deal with retries. Setting this larger will increase the speed 41530 slightly (at most 10% for 128 MiB in tests) at the cost of using more 41531 memory. It can be set smaller if you are tight on memory. 41532 41533 Properties: 41534 41535 - Config: chunk_size 41536 - Env Var: RCLONE_DROPBOX_CHUNK_SIZE 41537 - Type: SizeSuffix 41538 - Default: 48Mi 41539 41540 #### --dropbox-impersonate 41541 41542 Impersonate this user when using a business account. 41543 41544 Note that if you want to use impersonate, you should make sure this 41545 flag is set when running \[dq]rclone config\[dq] as this will cause rclone to 41546 request the \[dq]members.read\[dq] scope which it won\[aq]t normally. This is 41547 needed to lookup a members email address into the internal ID that 41548 dropbox uses in the API. 41549 41550 Using the \[dq]members.read\[dq] scope will require a Dropbox Team Admin 41551 to approve during the OAuth flow. 41552 41553 You will have to use your own App (setting your own client_id and 41554 client_secret) to use this option as currently rclone\[aq]s default set of 41555 permissions doesn\[aq]t include \[dq]members.read\[dq]. This can be added once 41556 v1.55 or later is in use everywhere. 41557 41558 41559 Properties: 41560 41561 - Config: impersonate 41562 - Env Var: RCLONE_DROPBOX_IMPERSONATE 41563 - Type: string 41564 - Required: false 41565 41566 #### --dropbox-shared-files 41567 41568 Instructs rclone to work on individual shared files. 41569 41570 In this mode rclone\[aq]s features are extremely limited - only list (ls, lsl, etc.) 41571 operations and read operations (e.g. downloading) are supported in this mode. 41572 All other operations will be disabled. 41573 41574 Properties: 41575 41576 - Config: shared_files 41577 - Env Var: RCLONE_DROPBOX_SHARED_FILES 41578 - Type: bool 41579 - Default: false 41580 41581 #### --dropbox-shared-folders 41582 41583 Instructs rclone to work on shared folders. 41584 41585 When this flag is used with no path only the List operation is supported and 41586 all available shared folders will be listed. If you specify a path the first part 41587 will be interpreted as the name of shared folder. Rclone will then try to mount this 41588 shared to the root namespace. On success shared folder rclone proceeds normally. 41589 The shared folder is now pretty much a normal folder and all normal operations 41590 are supported. 41591 41592 Note that we don\[aq]t unmount the shared folder afterwards so the 41593 --dropbox-shared-folders can be omitted after the first use of a particular 41594 shared folder. 41595 41596 Properties: 41597 41598 - Config: shared_folders 41599 - Env Var: RCLONE_DROPBOX_SHARED_FOLDERS 41600 - Type: bool 41601 - Default: false 41602 41603 #### --dropbox-pacer-min-sleep 41604 41605 Minimum time to sleep between API calls. 41606 41607 Properties: 41608 41609 - Config: pacer_min_sleep 41610 - Env Var: RCLONE_DROPBOX_PACER_MIN_SLEEP 41611 - Type: Duration 41612 - Default: 10ms 41613 41614 #### --dropbox-encoding 41615 41616 The encoding for the backend. 41617 41618 See the [encoding section in the overview](https://rclone.org/overview/#encoding) for more info. 41619 41620 Properties: 41621 41622 - Config: encoding 41623 - Env Var: RCLONE_DROPBOX_ENCODING 41624 - Type: Encoding 41625 - Default: Slash,BackSlash,Del,RightSpace,InvalidUtf8,Dot 41626 41627 #### --dropbox-batch-mode 41628 41629 Upload file batching sync|async|off. 41630 41631 This sets the batch mode used by rclone. 41632 41633 For full info see [the main docs](https://rclone.org/dropbox/#batch-mode) 41634 41635 This has 3 possible values 41636 41637 - off - no batching 41638 - sync - batch uploads and check completion (default) 41639 - async - batch upload and don\[aq]t check completion 41640 41641 Rclone will close any outstanding batches when it exits which may make 41642 a delay on quit. 41643 41644 41645 Properties: 41646 41647 - Config: batch_mode 41648 - Env Var: RCLONE_DROPBOX_BATCH_MODE 41649 - Type: string 41650 - Default: \[dq]sync\[dq] 41651 41652 #### --dropbox-batch-size 41653 41654 Max number of files in upload batch. 41655 41656 This sets the batch size of files to upload. It has to be less than 1000. 41657 41658 By default this is 0 which means rclone which calculate the batch size 41659 depending on the setting of batch_mode. 41660 41661 - batch_mode: async - default batch_size is 100 41662 - batch_mode: sync - default batch_size is the same as --transfers 41663 - batch_mode: off - not in use 41664 41665 Rclone will close any outstanding batches when it exits which may make 41666 a delay on quit. 41667 41668 Setting this is a great idea if you are uploading lots of small files 41669 as it will make them a lot quicker. You can use --transfers 32 to 41670 maximise throughput. 41671 41672 41673 Properties: 41674 41675 - Config: batch_size 41676 - Env Var: RCLONE_DROPBOX_BATCH_SIZE 41677 - Type: int 41678 - Default: 0 41679 41680 #### --dropbox-batch-timeout 41681 41682 Max time to allow an idle upload batch before uploading. 41683 41684 If an upload batch is idle for more than this long then it will be 41685 uploaded. 41686 41687 The default for this is 0 which means rclone will choose a sensible 41688 default based on the batch_mode in use. 41689 41690 - batch_mode: async - default batch_timeout is 10s 41691 - batch_mode: sync - default batch_timeout is 500ms 41692 - batch_mode: off - not in use 41693 41694 41695 Properties: 41696 41697 - Config: batch_timeout 41698 - Env Var: RCLONE_DROPBOX_BATCH_TIMEOUT 41699 - Type: Duration 41700 - Default: 0s 41701 41702 #### --dropbox-batch-commit-timeout 41703 41704 Max time to wait for a batch to finish committing 41705 41706 Properties: 41707 41708 - Config: batch_commit_timeout 41709 - Env Var: RCLONE_DROPBOX_BATCH_COMMIT_TIMEOUT 41710 - Type: Duration 41711 - Default: 10m0s 41712 41713 #### --dropbox-description 41714 41715 Description of the remote 41716 41717 Properties: 41718 41719 - Config: description 41720 - Env Var: RCLONE_DROPBOX_DESCRIPTION 41721 - Type: string 41722 - Required: false 41723 41724 41725 41726 ## Limitations 41727 41728 Note that Dropbox is case insensitive so you can\[aq]t have a file called 41729 \[dq]Hello.doc\[dq] and one called \[dq]hello.doc\[dq]. 41730 41731 There are some file names such as \[ga]thumbs.db\[ga] which Dropbox can\[aq]t 41732 store. There is a full list of them in the [\[dq]Ignored Files\[dq] section 41733 of this document](https://www.dropbox.com/en/help/145). Rclone will 41734 issue an error message \[ga]File name disallowed - not uploading\[ga] if it 41735 attempts to upload one of those file names, but the sync won\[aq]t fail. 41736 41737 Some errors may occur if you try to sync copyright-protected files 41738 because Dropbox has its own [copyright detector](https://techcrunch.com/2014/03/30/how-dropbox-knows-when-youre-sharing-copyrighted-stuff-without-actually-looking-at-your-stuff/) that 41739 prevents this sort of file being downloaded. This will return the error \[ga]ERROR : 41740 /path/to/your/file: Failed to copy: failed to open source object: 41741 path/restricted_content/.\[ga] 41742 41743 If you have more than 10,000 files in a directory then \[ga]rclone purge 41744 dropbox:dir\[ga] will return the error \[ga]Failed to purge: There are too 41745 many files involved in this operation\[ga]. As a work-around do an 41746 \[ga]rclone delete dropbox:dir\[ga] followed by an \[ga]rclone rmdir dropbox:dir\[ga]. 41747 41748 When using \[ga]rclone link\[ga] you\[aq]ll need to set \[ga]--expire\[ga] if using a 41749 non-personal account otherwise the visibility may not be correct. 41750 (Note that \[ga]--expire\[ga] isn\[aq]t supported on personal accounts). See the 41751 [forum discussion](https://forum.rclone.org/t/rclone-link-dropbox-permissions/23211) and the 41752 [dropbox SDK issue](https://github.com/dropbox/dropbox-sdk-go-unofficial/issues/75). 41753 41754 ## Get your own Dropbox App ID 41755 41756 When you use rclone with Dropbox in its default configuration you are using rclone\[aq]s App ID. This is shared between all the rclone users. 41757 41758 Here is how to create your own Dropbox App ID for rclone: 41759 41760 1. Log into the [Dropbox App console](https://www.dropbox.com/developers/apps/create) with your Dropbox Account (It need not 41761 to be the same account as the Dropbox you want to access) 41762 41763 2. Choose an API => Usually this should be \[ga]Dropbox API\[ga] 41764 41765 3. Choose the type of access you want to use => \[ga]Full Dropbox\[ga] or \[ga]App Folder\[ga]. If you want to use Team Folders, \[ga]Full Dropbox\[ga] is required ([see here](https://www.dropboxforum.com/t5/Dropbox-API-Support-Feedback/How-to-create-team-folder-inside-my-app-s-folder/m-p/601005/highlight/true#M27911)). 41766 41767 4. Name your App. The app name is global, so you can\[aq]t use \[ga]rclone\[ga] for example 41768 41769 5. Click the button \[ga]Create App\[ga] 41770 41771 6. Switch to the \[ga]Permissions\[ga] tab. Enable at least the following permissions: \[ga]account_info.read\[ga], \[ga]files.metadata.write\[ga], \[ga]files.content.write\[ga], \[ga]files.content.read\[ga], \[ga]sharing.write\[ga]. The \[ga]files.metadata.read\[ga] and \[ga]sharing.read\[ga] checkboxes will be marked too. Click \[ga]Submit\[ga] 41772 41773 7. Switch to the \[ga]Settings\[ga] tab. Fill \[ga]OAuth2 - Redirect URIs\[ga] as \[ga]http://localhost:53682/\[ga] and click on \[ga]Add\[ga] 41774 41775 8. Find the \[ga]App key\[ga] and \[ga]App secret\[ga] values on the \[ga]Settings\[ga] tab. Use these values in rclone config to add a new remote or edit an existing remote. The \[ga]App key\[ga] setting corresponds to \[ga]client_id\[ga] in rclone config, the \[ga]App secret\[ga] corresponds to \[ga]client_secret\[ga] 41776 41777 # Enterprise File Fabric 41778 41779 This backend supports [Storage Made Easy\[aq]s Enterprise File 41780 Fabric\[tm]](https://storagemadeeasy.com/about/) which provides a software 41781 solution to integrate and unify File and Object Storage accessible 41782 through a global file system. 41783 41784 ## Configuration 41785 41786 The initial setup for the Enterprise File Fabric backend involves 41787 getting a token from the Enterprise File Fabric which you need to 41788 do in your browser. \[ga]rclone config\[ga] walks you through it. 41789 41790 Here is an example of how to make a remote called \[ga]remote\[ga]. First run: 41791 41792 rclone config 41793 41794 This will guide you through an interactive setup process: 41795 \f[R] 41796 .fi 41797 .PP 41798 No remotes found, make a new one? 41799 n) New remote s) Set configuration password q) Quit config n/s/q> n 41800 name> remote Type of storage to configure. 41801 Enter a string value. 41802 Press Enter for the default (\[dq]\[dq]). 41803 Choose a number from below, or type in your own value [snip] XX / 41804 Enterprise File Fabric \ \[dq]filefabric\[dq] [snip] Storage> filefabric 41805 ** See help for filefabric backend at: https://rclone.org/filefabric/ ** 41806 .PP 41807 URL of the Enterprise File Fabric to connect to Enter a string value. 41808 Press Enter for the default (\[dq]\[dq]). 41809 Choose a number from below, or type in your own value 1 / Storage Made 41810 Easy US \ \[dq]https://storagemadeeasy.com\[dq] 2 / Storage Made Easy EU 41811 \ \[dq]https://eu.storagemadeeasy.com\[dq] 3 / Connect to your 41812 Enterprise File Fabric \ \[dq]https://yourfabric.smestorage.com\[dq] 41813 url> https://yourfabric.smestorage.com/ ID of the root folder Leave 41814 blank normally. 41815 .PP 41816 Fill in to make rclone start with directory of a given ID. 41817 .PP 41818 Enter a string value. 41819 Press Enter for the default (\[dq]\[dq]). 41820 root_folder_id> Permanent Authentication Token 41821 .PP 41822 A Permanent Authentication Token can be created in the Enterprise File 41823 Fabric, on the users Dashboard under Security, there is an entry 41824 you\[aq]ll see called \[dq]My Authentication Tokens\[dq]. 41825 Click the Manage button to create one. 41826 .PP 41827 These tokens are normally valid for several years. 41828 .PP 41829 For more info see: 41830 https://docs.storagemadeeasy.com/organisationcloud/api-tokens 41831 .PP 41832 Enter a string value. 41833 Press Enter for the default (\[dq]\[dq]). 41834 permanent_token> xxxxxxxxxxxxxxx-xxxxxxxxxxxxxxxx Edit advanced config? 41835 (y/n) y) Yes n) No (default) y/n> n Remote config -------------------- 41836 [remote] type = filefabric url = https://yourfabric.smestorage.com/ 41837 permanent_token = xxxxxxxxxxxxxxx-xxxxxxxxxxxxxxxx -------------------- 41838 y) Yes this is OK (default) e) Edit this remote d) Delete this remote 41839 y/e/d> y 41840 .IP 41841 .nf 41842 \f[C] 41843 Once configured you can then use \[ga]rclone\[ga] like this, 41844 41845 List directories in top level of your Enterprise File Fabric 41846 41847 rclone lsd remote: 41848 41849 List all the files in your Enterprise File Fabric 41850 41851 rclone ls remote: 41852 41853 To copy a local directory to an Enterprise File Fabric directory called backup 41854 41855 rclone copy /home/source remote:backup 41856 41857 ### Modification times and hashes 41858 41859 The Enterprise File Fabric allows modification times to be set on 41860 files accurate to 1 second. These will be used to detect whether 41861 objects need syncing or not. 41862 41863 The Enterprise File Fabric does not support any data hashes at this time. 41864 41865 ### Restricted filename characters 41866 41867 The [default restricted characters set](https://rclone.org/overview/#restricted-characters) 41868 will be replaced. 41869 41870 Invalid UTF-8 bytes will also be [replaced](https://rclone.org/overview/#invalid-utf8), 41871 as they can\[aq]t be used in JSON strings. 41872 41873 ### Empty files 41874 41875 Empty files aren\[aq]t supported by the Enterprise File Fabric. Rclone will therefore 41876 upload an empty file as a single space with a mime type of 41877 \[ga]application/vnd.rclone.empty.file\[ga] and files with that mime type are 41878 treated as empty. 41879 41880 ### Root folder ID ### 41881 41882 You can set the \[ga]root_folder_id\[ga] for rclone. This is the directory 41883 (identified by its \[ga]Folder ID\[ga]) that rclone considers to be the root 41884 of your Enterprise File Fabric. 41885 41886 Normally you will leave this blank and rclone will determine the 41887 correct root to use itself. 41888 41889 However you can set this to restrict rclone to a specific folder 41890 hierarchy. 41891 41892 In order to do this you will have to find the \[ga]Folder ID\[ga] of the 41893 directory you wish rclone to display. These aren\[aq]t displayed in the 41894 web interface, but you can use \[ga]rclone lsf\[ga] to find them, for example 41895 \f[R] 41896 .fi 41897 .PP 41898 $ rclone lsf --dirs-only -Fip --csv filefabric: 120673758,Burnt PDFs/ 41899 120673759,My Quick Uploads/ 120673755,My Syncs/ 120673756,My backups/ 41900 120673757,My contacts/ 120673761,S3 Storage/ 41901 .IP 41902 .nf 41903 \f[C] 41904 The ID for \[dq]S3 Storage\[dq] would be \[ga]120673761\[ga]. 41905 41906 41907 ### Standard options 41908 41909 Here are the Standard options specific to filefabric (Enterprise File Fabric). 41910 41911 #### --filefabric-url 41912 41913 URL of the Enterprise File Fabric to connect to. 41914 41915 Properties: 41916 41917 - Config: url 41918 - Env Var: RCLONE_FILEFABRIC_URL 41919 - Type: string 41920 - Required: true 41921 - Examples: 41922 - \[dq]https://storagemadeeasy.com\[dq] 41923 - Storage Made Easy US 41924 - \[dq]https://eu.storagemadeeasy.com\[dq] 41925 - Storage Made Easy EU 41926 - \[dq]https://yourfabric.smestorage.com\[dq] 41927 - Connect to your Enterprise File Fabric 41928 41929 #### --filefabric-root-folder-id 41930 41931 ID of the root folder. 41932 41933 Leave blank normally. 41934 41935 Fill in to make rclone start with directory of a given ID. 41936 41937 41938 Properties: 41939 41940 - Config: root_folder_id 41941 - Env Var: RCLONE_FILEFABRIC_ROOT_FOLDER_ID 41942 - Type: string 41943 - Required: false 41944 41945 #### --filefabric-permanent-token 41946 41947 Permanent Authentication Token. 41948 41949 A Permanent Authentication Token can be created in the Enterprise File 41950 Fabric, on the users Dashboard under Security, there is an entry 41951 you\[aq]ll see called \[dq]My Authentication Tokens\[dq]. Click the Manage button 41952 to create one. 41953 41954 These tokens are normally valid for several years. 41955 41956 For more info see: https://docs.storagemadeeasy.com/organisationcloud/api-tokens 41957 41958 41959 Properties: 41960 41961 - Config: permanent_token 41962 - Env Var: RCLONE_FILEFABRIC_PERMANENT_TOKEN 41963 - Type: string 41964 - Required: false 41965 41966 ### Advanced options 41967 41968 Here are the Advanced options specific to filefabric (Enterprise File Fabric). 41969 41970 #### --filefabric-token 41971 41972 Session Token. 41973 41974 This is a session token which rclone caches in the config file. It is 41975 usually valid for 1 hour. 41976 41977 Don\[aq]t set this value - rclone will set it automatically. 41978 41979 41980 Properties: 41981 41982 - Config: token 41983 - Env Var: RCLONE_FILEFABRIC_TOKEN 41984 - Type: string 41985 - Required: false 41986 41987 #### --filefabric-token-expiry 41988 41989 Token expiry time. 41990 41991 Don\[aq]t set this value - rclone will set it automatically. 41992 41993 41994 Properties: 41995 41996 - Config: token_expiry 41997 - Env Var: RCLONE_FILEFABRIC_TOKEN_EXPIRY 41998 - Type: string 41999 - Required: false 42000 42001 #### --filefabric-version 42002 42003 Version read from the file fabric. 42004 42005 Don\[aq]t set this value - rclone will set it automatically. 42006 42007 42008 Properties: 42009 42010 - Config: version 42011 - Env Var: RCLONE_FILEFABRIC_VERSION 42012 - Type: string 42013 - Required: false 42014 42015 #### --filefabric-encoding 42016 42017 The encoding for the backend. 42018 42019 See the [encoding section in the overview](https://rclone.org/overview/#encoding) for more info. 42020 42021 Properties: 42022 42023 - Config: encoding 42024 - Env Var: RCLONE_FILEFABRIC_ENCODING 42025 - Type: Encoding 42026 - Default: Slash,Del,Ctl,InvalidUtf8,Dot 42027 42028 #### --filefabric-description 42029 42030 Description of the remote 42031 42032 Properties: 42033 42034 - Config: description 42035 - Env Var: RCLONE_FILEFABRIC_DESCRIPTION 42036 - Type: string 42037 - Required: false 42038 42039 42040 42041 # FTP 42042 42043 FTP is the File Transfer Protocol. Rclone FTP support is provided using the 42044 [github.com/jlaffaye/ftp](https://godoc.org/github.com/jlaffaye/ftp) 42045 package. 42046 42047 [Limitations of Rclone\[aq]s FTP backend](#limitations) 42048 42049 Paths are specified as \[ga]remote:path\[ga]. If the path does not begin with 42050 a \[ga]/\[ga] it is relative to the home directory of the user. An empty path 42051 \[ga]remote:\[ga] refers to the user\[aq]s home directory. 42052 42053 ## Configuration 42054 42055 To create an FTP configuration named \[ga]remote\[ga], run 42056 42057 rclone config 42058 42059 Rclone config guides you through an interactive setup process. A minimal 42060 rclone FTP remote definition only requires host, username and password. 42061 For an anonymous FTP server, see [below](#anonymous-ftp). 42062 \f[R] 42063 .fi 42064 .PP 42065 No remotes found, make a new one? 42066 n) New remote r) Rename remote c) Copy remote s) Set configuration 42067 password q) Quit config n/r/c/s/q> n name> remote Type of storage to 42068 configure. 42069 Enter a string value. 42070 Press Enter for the default (\[dq]\[dq]). 42071 Choose a number from below, or type in your own value [snip] XX / FTP 42072 \ \[dq]ftp\[dq] [snip] Storage> ftp ** See help for ftp backend at: 42073 https://rclone.org/ftp/ ** 42074 .PP 42075 FTP host to connect to Enter a string value. 42076 Press Enter for the default (\[dq]\[dq]). 42077 Choose a number from below, or type in your own value 1 / Connect to 42078 ftp.example.com \ \[dq]ftp.example.com\[dq] host> ftp.example.com FTP 42079 username Enter a string value. 42080 Press Enter for the default (\[dq]$USER\[dq]). 42081 user> FTP port number Enter a signed integer. 42082 Press Enter for the default (21). 42083 port> FTP password y) Yes type in my own password g) Generate random 42084 password y/g> y Enter the password: password: Confirm the password: 42085 password: Use FTP over TLS (Implicit) Enter a boolean value (true or 42086 false). 42087 Press Enter for the default (\[dq]false\[dq]). 42088 tls> Use FTP over TLS (Explicit) Enter a boolean value (true or false). 42089 Press Enter for the default (\[dq]false\[dq]). 42090 explicit_tls> Remote config -------------------- [remote] type = ftp 42091 host = ftp.example.com pass = *** ENCRYPTED *** -------------------- y) 42092 Yes this is OK e) Edit this remote d) Delete this remote y/e/d> y 42093 .IP 42094 .nf 42095 \f[C] 42096 To see all directories in the home directory of \[ga]remote\[ga] 42097 42098 rclone lsd remote: 42099 42100 Make a new directory 42101 42102 rclone mkdir remote:path/to/directory 42103 42104 List the contents of a directory 42105 42106 rclone ls remote:path/to/directory 42107 42108 Sync \[ga]/home/local/directory\[ga] to the remote directory, deleting any 42109 excess files in the directory. 42110 42111 rclone sync --interactive /home/local/directory remote:directory 42112 42113 ### Anonymous FTP 42114 42115 When connecting to a FTP server that allows anonymous login, you can use the 42116 special \[dq]anonymous\[dq] username. Traditionally, this user account accepts any 42117 string as a password, although it is common to use either the password 42118 \[dq]anonymous\[dq] or \[dq]guest\[dq]. Some servers require the use of a valid e-mail 42119 address as password. 42120 42121 Using [on-the-fly](#backend-path-to-dir) or 42122 [connection string](https://rclone.org/docs/#connection-strings) remotes makes it easy to access 42123 such servers, without requiring any configuration in advance. The following 42124 are examples of that: 42125 42126 rclone lsf :ftp: --ftp-host=speedtest.tele2.net --ftp-user=anonymous --ftp-pass=$(rclone obscure dummy) 42127 rclone lsf :ftp,host=speedtest.tele2.net,user=anonymous,pass=$(rclone obscure dummy): 42128 42129 The above examples work in Linux shells and in PowerShell, but not Windows 42130 Command Prompt. They execute the [rclone obscure](https://rclone.org/commands/rclone_obscure/) 42131 command to create a password string in the format required by the 42132 [pass](#ftp-pass) option. The following examples are exactly the same, except use 42133 an already obscured string representation of the same password \[dq]dummy\[dq], and 42134 therefore works even in Windows Command Prompt: 42135 42136 rclone lsf :ftp: --ftp-host=speedtest.tele2.net --ftp-user=anonymous --ftp-pass=IXs2wc8OJOz7SYLBk47Ji1rHTmxM 42137 rclone lsf :ftp,host=speedtest.tele2.net,user=anonymous,pass=IXs2wc8OJOz7SYLBk47Ji1rHTmxM: 42138 42139 ### Implicit TLS 42140 42141 Rlone FTP supports implicit FTP over TLS servers (FTPS). This has to 42142 be enabled in the FTP backend config for the remote, or with 42143 [\[ga]--ftp-tls\[ga]](#ftp-tls). The default FTPS port is \[ga]990\[ga], not \[ga]21\[ga] and 42144 can be set with [\[ga]--ftp-port\[ga]](#ftp-port). 42145 42146 ### Restricted filename characters 42147 42148 In addition to the [default restricted characters set](https://rclone.org/overview/#restricted-characters) 42149 the following characters are also replaced: 42150 42151 File names cannot end with the following characters. Replacement is 42152 limited to the last character in a file name: 42153 42154 | Character | Value | Replacement | 42155 | --------- |:-----:|:-----------:| 42156 | SP | 0x20 | \[u2420] | 42157 42158 Not all FTP servers can have all characters in file names, for example: 42159 42160 | FTP Server| Forbidden characters | 42161 | --------- |:--------------------:| 42162 | proftpd | \[ga]*\[ga] | 42163 | pureftpd | \[ga]\[rs] [ ]\[ga] | 42164 42165 This backend\[aq]s interactive configuration wizard provides a selection of 42166 sensible encoding settings for major FTP servers: ProFTPd, PureFTPd, VsFTPd. 42167 Just hit a selection number when prompted. 42168 42169 42170 ### Standard options 42171 42172 Here are the Standard options specific to ftp (FTP). 42173 42174 #### --ftp-host 42175 42176 FTP host to connect to. 42177 42178 E.g. \[dq]ftp.example.com\[dq]. 42179 42180 Properties: 42181 42182 - Config: host 42183 - Env Var: RCLONE_FTP_HOST 42184 - Type: string 42185 - Required: true 42186 42187 #### --ftp-user 42188 42189 FTP username. 42190 42191 Properties: 42192 42193 - Config: user 42194 - Env Var: RCLONE_FTP_USER 42195 - Type: string 42196 - Default: \[dq]$USER\[dq] 42197 42198 #### --ftp-port 42199 42200 FTP port number. 42201 42202 Properties: 42203 42204 - Config: port 42205 - Env Var: RCLONE_FTP_PORT 42206 - Type: int 42207 - Default: 21 42208 42209 #### --ftp-pass 42210 42211 FTP password. 42212 42213 **NB** Input to this must be obscured - see [rclone obscure](https://rclone.org/commands/rclone_obscure/). 42214 42215 Properties: 42216 42217 - Config: pass 42218 - Env Var: RCLONE_FTP_PASS 42219 - Type: string 42220 - Required: false 42221 42222 #### --ftp-tls 42223 42224 Use Implicit FTPS (FTP over TLS). 42225 42226 When using implicit FTP over TLS the client connects using TLS 42227 right from the start which breaks compatibility with 42228 non-TLS-aware servers. This is usually served over port 990 rather 42229 than port 21. Cannot be used in combination with explicit FTPS. 42230 42231 Properties: 42232 42233 - Config: tls 42234 - Env Var: RCLONE_FTP_TLS 42235 - Type: bool 42236 - Default: false 42237 42238 #### --ftp-explicit-tls 42239 42240 Use Explicit FTPS (FTP over TLS). 42241 42242 When using explicit FTP over TLS the client explicitly requests 42243 security from the server in order to upgrade a plain text connection 42244 to an encrypted one. Cannot be used in combination with implicit FTPS. 42245 42246 Properties: 42247 42248 - Config: explicit_tls 42249 - Env Var: RCLONE_FTP_EXPLICIT_TLS 42250 - Type: bool 42251 - Default: false 42252 42253 ### Advanced options 42254 42255 Here are the Advanced options specific to ftp (FTP). 42256 42257 #### --ftp-concurrency 42258 42259 Maximum number of FTP simultaneous connections, 0 for unlimited. 42260 42261 Note that setting this is very likely to cause deadlocks so it should 42262 be used with care. 42263 42264 If you are doing a sync or copy then make sure concurrency is one more 42265 than the sum of \[ga]--transfers\[ga] and \[ga]--checkers\[ga]. 42266 42267 If you use \[ga]--check-first\[ga] then it just needs to be one more than the 42268 maximum of \[ga]--checkers\[ga] and \[ga]--transfers\[ga]. 42269 42270 So for \[ga]concurrency 3\[ga] you\[aq]d use \[ga]--checkers 2 --transfers 2 42271 --check-first\[ga] or \[ga]--checkers 1 --transfers 1\[ga]. 42272 42273 42274 42275 Properties: 42276 42277 - Config: concurrency 42278 - Env Var: RCLONE_FTP_CONCURRENCY 42279 - Type: int 42280 - Default: 0 42281 42282 #### --ftp-no-check-certificate 42283 42284 Do not verify the TLS certificate of the server. 42285 42286 Properties: 42287 42288 - Config: no_check_certificate 42289 - Env Var: RCLONE_FTP_NO_CHECK_CERTIFICATE 42290 - Type: bool 42291 - Default: false 42292 42293 #### --ftp-disable-epsv 42294 42295 Disable using EPSV even if server advertises support. 42296 42297 Properties: 42298 42299 - Config: disable_epsv 42300 - Env Var: RCLONE_FTP_DISABLE_EPSV 42301 - Type: bool 42302 - Default: false 42303 42304 #### --ftp-disable-mlsd 42305 42306 Disable using MLSD even if server advertises support. 42307 42308 Properties: 42309 42310 - Config: disable_mlsd 42311 - Env Var: RCLONE_FTP_DISABLE_MLSD 42312 - Type: bool 42313 - Default: false 42314 42315 #### --ftp-disable-utf8 42316 42317 Disable using UTF-8 even if server advertises support. 42318 42319 Properties: 42320 42321 - Config: disable_utf8 42322 - Env Var: RCLONE_FTP_DISABLE_UTF8 42323 - Type: bool 42324 - Default: false 42325 42326 #### --ftp-writing-mdtm 42327 42328 Use MDTM to set modification time (VsFtpd quirk) 42329 42330 Properties: 42331 42332 - Config: writing_mdtm 42333 - Env Var: RCLONE_FTP_WRITING_MDTM 42334 - Type: bool 42335 - Default: false 42336 42337 #### --ftp-force-list-hidden 42338 42339 Use LIST -a to force listing of hidden files and folders. This will disable the use of MLSD. 42340 42341 Properties: 42342 42343 - Config: force_list_hidden 42344 - Env Var: RCLONE_FTP_FORCE_LIST_HIDDEN 42345 - Type: bool 42346 - Default: false 42347 42348 #### --ftp-idle-timeout 42349 42350 Max time before closing idle connections. 42351 42352 If no connections have been returned to the connection pool in the time 42353 given, rclone will empty the connection pool. 42354 42355 Set to 0 to keep connections indefinitely. 42356 42357 42358 Properties: 42359 42360 - Config: idle_timeout 42361 - Env Var: RCLONE_FTP_IDLE_TIMEOUT 42362 - Type: Duration 42363 - Default: 1m0s 42364 42365 #### --ftp-close-timeout 42366 42367 Maximum time to wait for a response to close. 42368 42369 Properties: 42370 42371 - Config: close_timeout 42372 - Env Var: RCLONE_FTP_CLOSE_TIMEOUT 42373 - Type: Duration 42374 - Default: 1m0s 42375 42376 #### --ftp-tls-cache-size 42377 42378 Size of TLS session cache for all control and data connections. 42379 42380 TLS cache allows to resume TLS sessions and reuse PSK between connections. 42381 Increase if default size is not enough resulting in TLS resumption errors. 42382 Enabled by default. Use 0 to disable. 42383 42384 Properties: 42385 42386 - Config: tls_cache_size 42387 - Env Var: RCLONE_FTP_TLS_CACHE_SIZE 42388 - Type: int 42389 - Default: 32 42390 42391 #### --ftp-disable-tls13 42392 42393 Disable TLS 1.3 (workaround for FTP servers with buggy TLS) 42394 42395 Properties: 42396 42397 - Config: disable_tls13 42398 - Env Var: RCLONE_FTP_DISABLE_TLS13 42399 - Type: bool 42400 - Default: false 42401 42402 #### --ftp-shut-timeout 42403 42404 Maximum time to wait for data connection closing status. 42405 42406 Properties: 42407 42408 - Config: shut_timeout 42409 - Env Var: RCLONE_FTP_SHUT_TIMEOUT 42410 - Type: Duration 42411 - Default: 1m0s 42412 42413 #### --ftp-ask-password 42414 42415 Allow asking for FTP password when needed. 42416 42417 If this is set and no password is supplied then rclone will ask for a password 42418 42419 42420 Properties: 42421 42422 - Config: ask_password 42423 - Env Var: RCLONE_FTP_ASK_PASSWORD 42424 - Type: bool 42425 - Default: false 42426 42427 #### --ftp-socks-proxy 42428 42429 Socks 5 proxy host. 42430 42431 Supports the format user:pass\[at]host:port, user\[at]host:port, host:port. 42432 42433 Example: 42434 42435 myUser:myPass\[at]localhost:9005 42436 42437 42438 Properties: 42439 42440 - Config: socks_proxy 42441 - Env Var: RCLONE_FTP_SOCKS_PROXY 42442 - Type: string 42443 - Required: false 42444 42445 #### --ftp-encoding 42446 42447 The encoding for the backend. 42448 42449 See the [encoding section in the overview](https://rclone.org/overview/#encoding) for more info. 42450 42451 Properties: 42452 42453 - Config: encoding 42454 - Env Var: RCLONE_FTP_ENCODING 42455 - Type: Encoding 42456 - Default: Slash,Del,Ctl,RightSpace,Dot 42457 - Examples: 42458 - \[dq]Asterisk,Ctl,Dot,Slash\[dq] 42459 - ProFTPd can\[aq]t handle \[aq]*\[aq] in file names 42460 - \[dq]BackSlash,Ctl,Del,Dot,RightSpace,Slash,SquareBracket\[dq] 42461 - PureFTPd can\[aq]t handle \[aq][]\[aq] or \[aq]*\[aq] in file names 42462 - \[dq]Ctl,LeftPeriod,Slash\[dq] 42463 - VsFTPd can\[aq]t handle file names starting with dot 42464 42465 #### --ftp-description 42466 42467 Description of the remote 42468 42469 Properties: 42470 42471 - Config: description 42472 - Env Var: RCLONE_FTP_DESCRIPTION 42473 - Type: string 42474 - Required: false 42475 42476 42477 42478 ## Limitations 42479 42480 FTP servers acting as rclone remotes must support \[ga]passive\[ga] mode. 42481 The mode cannot be configured as \[ga]passive\[ga] is the only supported one. 42482 Rclone\[aq]s FTP implementation is not compatible with \[ga]active\[ga] mode 42483 as [the library it uses doesn\[aq]t support it](https://github.com/jlaffaye/ftp/issues/29). 42484 This will likely never be supported due to security concerns. 42485 42486 Rclone\[aq]s FTP backend does not support any checksums but can compare 42487 file sizes. 42488 42489 \[ga]rclone about\[ga] is not supported by the FTP backend. Backends without 42490 this capability cannot determine free space for an rclone mount or 42491 use policy \[ga]mfs\[ga] (most free space) as a member of an rclone union 42492 remote. 42493 42494 See [List of backends that do not support rclone about](https://rclone.org/overview/#optional-features) and [rclone about](https://rclone.org/commands/rclone_about/) 42495 42496 The implementation of : \[ga]--dump headers\[ga], 42497 \[ga]--dump bodies\[ga], \[ga]--dump auth\[ga] for debugging isn\[aq]t the same as 42498 for rclone HTTP based backends - it has less fine grained control. 42499 42500 \[ga]--timeout\[ga] isn\[aq]t supported (but \[ga]--contimeout\[ga] is). 42501 42502 \[ga]--bind\[ga] isn\[aq]t supported. 42503 42504 Rclone\[aq]s FTP backend could support server-side move but does not 42505 at present. 42506 42507 The \[ga]ftp_proxy\[ga] environment variable is not currently supported. 42508 42509 ### Modification times 42510 42511 File modification time (timestamps) is supported to 1 second resolution 42512 for major FTP servers: ProFTPd, PureFTPd, VsFTPd, and FileZilla FTP server. 42513 The \[ga]VsFTPd\[ga] server has non-standard implementation of time related protocol 42514 commands and needs a special configuration setting: \[ga]writing_mdtm = true\[ga]. 42515 42516 Support for precise file time with other FTP servers varies depending on what 42517 protocol extensions they advertise. If all the \[ga]MLSD\[ga], \[ga]MDTM\[ga] and \[ga]MFTM\[ga] 42518 extensions are present, rclone will use them together to provide precise time. 42519 Otherwise the times you see on the FTP server through rclone are those of the 42520 last file upload. 42521 42522 You can use the following command to check whether rclone can use precise time 42523 with your FTP server: \[ga]rclone backend features your_ftp_remote:\[ga] (the trailing 42524 colon is important). Look for the number in the line tagged by \[ga]Precision\[ga] 42525 designating the remote time precision expressed as nanoseconds. A value of 42526 \[ga]1000000000\[ga] means that file time precision of 1 second is available. 42527 A value of \[ga]3153600000000000000\[ga] (or another large number) means \[dq]unsupported\[dq]. 42528 42529 # Google Cloud Storage 42530 42531 Paths are specified as \[ga]remote:bucket\[ga] (or \[ga]remote:\[ga] for the \[ga]lsd\[ga] 42532 command.) You may put subdirectories in too, e.g. \[ga]remote:bucket/path/to/dir\[ga]. 42533 42534 ## Configuration 42535 42536 The initial setup for google cloud storage involves getting a token from Google Cloud Storage 42537 which you need to do in your browser. \[ga]rclone config\[ga] walks you 42538 through it. 42539 42540 Here is an example of how to make a remote called \[ga]remote\[ga]. First run: 42541 42542 rclone config 42543 42544 This will guide you through an interactive setup process: 42545 \f[R] 42546 .fi 42547 .IP "n)" 3 42548 New remote 42549 .IP "o)" 3 42550 Delete remote 42551 .IP "p)" 3 42552 Quit config e/n/d/q> n name> remote Type of storage to configure. 42553 Choose a number from below, or type in your own value [snip] XX / Google 42554 Cloud Storage (this is not Google Drive) \ \[dq]google cloud 42555 storage\[dq] [snip] Storage> google cloud storage Google Application 42556 Client Id - leave blank normally. 42557 client_id> Google Application Client Secret - leave blank normally. 42558 client_secret> Project number optional - needed only for 42559 list/create/delete buckets - see your developer console. 42560 project_number> 12345678 Service Account Credentials JSON file path - 42561 needed only if you want use SA instead of interactive login. 42562 service_account_file> Access Control List for new objects. 42563 Choose a number from below, or type in your own value 1 / Object owner 42564 gets OWNER access, and all Authenticated Users get READER access. 42565 \ \[dq]authenticatedRead\[dq] 2 / Object owner gets OWNER access, and 42566 project team owners get OWNER access. 42567 \ \[dq]bucketOwnerFullControl\[dq] 3 / Object owner gets OWNER access, 42568 and project team owners get READER access. 42569 \ \[dq]bucketOwnerRead\[dq] 4 / Object owner gets OWNER access [default 42570 if left blank]. 42571 \ \[dq]private\[dq] 5 / Object owner gets OWNER access, and project team 42572 members get access according to their roles. 42573 \ \[dq]projectPrivate\[dq] 6 / Object owner gets OWNER access, and all 42574 Users get READER access. 42575 \ \[dq]publicRead\[dq] object_acl> 4 Access Control List for new 42576 buckets. 42577 Choose a number from below, or type in your own value 1 / Project team 42578 owners get OWNER access, and all Authenticated Users get READER access. 42579 \ \[dq]authenticatedRead\[dq] 2 / Project team owners get OWNER access 42580 [default if left blank]. 42581 \ \[dq]private\[dq] 3 / Project team members get access according to 42582 their roles. 42583 \ \[dq]projectPrivate\[dq] 4 / Project team owners get OWNER access, and 42584 all Users get READER access. 42585 \ \[dq]publicRead\[dq] 5 / Project team owners get OWNER access, and all 42586 Users get WRITER access. 42587 \ \[dq]publicReadWrite\[dq] bucket_acl> 2 Location for the newly created 42588 buckets. 42589 Choose a number from below, or type in your own value 1 / Empty for 42590 default location (US). 42591 \ \[dq]\[dq] 2 / Multi-regional location for Asia. 42592 \ \[dq]asia\[dq] 3 / Multi-regional location for Europe. 42593 \ \[dq]eu\[dq] 4 / Multi-regional location for United States. 42594 \ \[dq]us\[dq] 5 / Taiwan. 42595 \ \[dq]asia-east1\[dq] 6 / Tokyo. 42596 \ \[dq]asia-northeast1\[dq] 7 / Singapore. 42597 \ \[dq]asia-southeast1\[dq] 8 / Sydney. 42598 \ \[dq]australia-southeast1\[dq] 9 / Belgium. 42599 \ \[dq]europe-west1\[dq] 10 / London. 42600 \ \[dq]europe-west2\[dq] 11 / Iowa. 42601 \ \[dq]us-central1\[dq] 12 / South Carolina. 42602 \ \[dq]us-east1\[dq] 13 / Northern Virginia. 42603 \ \[dq]us-east4\[dq] 14 / Oregon. 42604 \ \[dq]us-west1\[dq] location> 12 The storage class to use when storing 42605 objects in Google Cloud Storage. 42606 Choose a number from below, or type in your own value 1 / Default 42607 \ \[dq]\[dq] 2 / Multi-regional storage class \ \[dq]MULTI_REGIONAL\[dq] 42608 3 / Regional storage class \ \[dq]REGIONAL\[dq] 4 / Nearline storage 42609 class \ \[dq]NEARLINE\[dq] 5 / Coldline storage class 42610 \ \[dq]COLDLINE\[dq] 6 / Durable reduced availability storage class 42611 \ \[dq]DURABLE_REDUCED_AVAILABILITY\[dq] storage_class> 5 Remote config 42612 Use web browser to automatically authenticate rclone with remote? 42613 .IP \[bu] 2 42614 Say Y if the machine running rclone has a web browser you can use 42615 .IP \[bu] 2 42616 Say N if running rclone on a (remote) machine without web browser access 42617 If not sure try Y. 42618 If Y failed, try N. 42619 .IP "y)" 3 42620 Yes 42621 .IP "z)" 3 42622 No y/n> y If your browser doesn\[aq]t open automatically go to the 42623 following link: http://127.0.0.1:53682/auth Log in and authorize rclone 42624 for access Waiting for code... 42625 Got code -------------------- [remote] type = google cloud storage 42626 client_id = client_secret = token = 42627 {\[dq]AccessToken\[dq]:\[dq]xxxx.xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\[dq],\[dq]RefreshToken\[dq]:\[dq]x/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_xxxxxxxxx\[dq],\[dq]Expiry\[dq]:\[dq]2014-07-17T20:49:14.929208288+01:00\[dq],\[dq]Extra\[dq]:null} 42628 project_number = 12345678 object_acl = private bucket_acl = private 42629 -------------------- 42630 .IP "a)" 3 42631 Yes this is OK 42632 .IP "b)" 3 42633 Edit this remote 42634 .IP "c)" 3 42635 Delete this remote y/e/d> y 42636 .IP 42637 .nf 42638 \f[C] 42639 See the [remote setup docs](https://rclone.org/remote_setup/) for how to set it up on a 42640 machine with no Internet browser available. 42641 42642 Note that rclone runs a webserver on your local machine to collect the 42643 token as returned from Google if using web browser to automatically 42644 authenticate. This only 42645 runs from the moment it opens your browser to the moment you get back 42646 the verification code. This is on \[ga]http://127.0.0.1:53682/\[ga] and this 42647 it may require you to unblock it temporarily if you are running a host 42648 firewall, or use manual mode. 42649 42650 This remote is called \[ga]remote\[ga] and can now be used like this 42651 42652 See all the buckets in your project 42653 42654 rclone lsd remote: 42655 42656 Make a new bucket 42657 42658 rclone mkdir remote:bucket 42659 42660 List the contents of a bucket 42661 42662 rclone ls remote:bucket 42663 42664 Sync \[ga]/home/local/directory\[ga] to the remote bucket, deleting any excess 42665 files in the bucket. 42666 42667 rclone sync --interactive /home/local/directory remote:bucket 42668 42669 ### Service Account support 42670 42671 You can set up rclone with Google Cloud Storage in an unattended mode, 42672 i.e. not tied to a specific end-user Google account. This is useful 42673 when you want to synchronise files onto machines that don\[aq]t have 42674 actively logged-in users, for example build machines. 42675 42676 To get credentials for Google Cloud Platform 42677 [IAM Service Accounts](https://cloud.google.com/iam/docs/service-accounts), 42678 please head to the 42679 [Service Account](https://console.cloud.google.com/permissions/serviceaccounts) 42680 section of the Google Developer Console. Service Accounts behave just 42681 like normal \[ga]User\[ga] permissions in 42682 [Google Cloud Storage ACLs](https://cloud.google.com/storage/docs/access-control), 42683 so you can limit their access (e.g. make them read only). After 42684 creating an account, a JSON file containing the Service Account\[aq]s 42685 credentials will be downloaded onto your machines. These credentials 42686 are what rclone will use for authentication. 42687 42688 To use a Service Account instead of OAuth2 token flow, enter the path 42689 to your Service Account credentials at the \[ga]service_account_file\[ga] 42690 prompt and rclone won\[aq]t use the browser based authentication 42691 flow. If you\[aq]d rather stuff the contents of the credentials file into 42692 the rclone config file, you can set \[ga]service_account_credentials\[ga] with 42693 the actual contents of the file instead, or set the equivalent 42694 environment variable. 42695 42696 ### Anonymous Access 42697 42698 For downloads of objects that permit public access you can configure rclone 42699 to use anonymous access by setting \[ga]anonymous\[ga] to \[ga]true\[ga]. 42700 With unauthorized access you can\[aq]t write or create files but only read or list 42701 those buckets and objects that have public read access. 42702 42703 ### Application Default Credentials 42704 42705 If no other source of credentials is provided, rclone will fall back 42706 to 42707 [Application Default Credentials](https://cloud.google.com/video-intelligence/docs/common/auth#authenticating_with_application_default_credentials) 42708 this is useful both when you already have configured authentication 42709 for your developer account, or in production when running on a google 42710 compute host. Note that if running in docker, you may need to run 42711 additional commands on your google compute machine - 42712 [see this page](https://cloud.google.com/container-registry/docs/advanced-authentication#gcloud_as_a_docker_credential_helper). 42713 42714 Note that in the case application default credentials are used, there 42715 is no need to explicitly configure a project number. 42716 42717 ### --fast-list 42718 42719 This remote supports \[ga]--fast-list\[ga] which allows you to use fewer 42720 transactions in exchange for more memory. See the [rclone 42721 docs](https://rclone.org/docs/#fast-list) for more details. 42722 42723 ### Custom upload headers 42724 42725 You can set custom upload headers with the \[ga]--header-upload\[ga] 42726 flag. Google Cloud Storage supports the headers as described in the 42727 [working with metadata documentation](https://cloud.google.com/storage/docs/gsutil/addlhelp/WorkingWithObjectMetadata) 42728 42729 - Cache-Control 42730 - Content-Disposition 42731 - Content-Encoding 42732 - Content-Language 42733 - Content-Type 42734 - X-Goog-Storage-Class 42735 - X-Goog-Meta- 42736 42737 Eg \[ga]--header-upload \[dq]Content-Type text/potato\[dq]\[ga] 42738 42739 Note that the last of these is for setting custom metadata in the form 42740 \[ga]--header-upload \[dq]x-goog-meta-key: value\[dq]\[ga] 42741 42742 ### Modification times 42743 42744 Google Cloud Storage stores md5sum natively. 42745 Google\[aq]s [gsutil](https://cloud.google.com/storage/docs/gsutil) tool stores modification time 42746 with one-second precision as \[ga]goog-reserved-file-mtime\[ga] in file metadata. 42747 42748 To ensure compatibility with gsutil, rclone stores modification time in 2 separate metadata entries. 42749 \[ga]mtime\[ga] uses RFC3339 format with one-nanosecond precision. 42750 \[ga]goog-reserved-file-mtime\[ga] uses Unix timestamp format with one-second precision. 42751 To get modification time from object metadata, rclone reads the metadata in the following order: \[ga]mtime\[ga], \[ga]goog-reserved-file-mtime\[ga], object updated time. 42752 42753 Note that rclone\[aq]s default modify window is 1ns. 42754 Files uploaded by gsutil only contain timestamps with one-second precision. 42755 If you use rclone to sync files previously uploaded by gsutil, 42756 rclone will attempt to update modification time for all these files. 42757 To avoid these possibly unnecessary updates, use \[ga]--modify-window 1s\[ga]. 42758 42759 ### Restricted filename characters 42760 42761 | Character | Value | Replacement | 42762 | --------- |:-----:|:-----------:| 42763 | NUL | 0x00 | \[u2400] | 42764 | LF | 0x0A | \[u240A] | 42765 | CR | 0x0D | \[u240D] | 42766 | / | 0x2F | \[uFF0F] | 42767 42768 Invalid UTF-8 bytes will also be [replaced](https://rclone.org/overview/#invalid-utf8), 42769 as they can\[aq]t be used in JSON strings. 42770 42771 42772 ### Standard options 42773 42774 Here are the Standard options specific to google cloud storage (Google Cloud Storage (this is not Google Drive)). 42775 42776 #### --gcs-client-id 42777 42778 OAuth Client Id. 42779 42780 Leave blank normally. 42781 42782 Properties: 42783 42784 - Config: client_id 42785 - Env Var: RCLONE_GCS_CLIENT_ID 42786 - Type: string 42787 - Required: false 42788 42789 #### --gcs-client-secret 42790 42791 OAuth Client Secret. 42792 42793 Leave blank normally. 42794 42795 Properties: 42796 42797 - Config: client_secret 42798 - Env Var: RCLONE_GCS_CLIENT_SECRET 42799 - Type: string 42800 - Required: false 42801 42802 #### --gcs-project-number 42803 42804 Project number. 42805 42806 Optional - needed only for list/create/delete buckets - see your developer console. 42807 42808 Properties: 42809 42810 - Config: project_number 42811 - Env Var: RCLONE_GCS_PROJECT_NUMBER 42812 - Type: string 42813 - Required: false 42814 42815 #### --gcs-user-project 42816 42817 User project. 42818 42819 Optional - needed only for requester pays. 42820 42821 Properties: 42822 42823 - Config: user_project 42824 - Env Var: RCLONE_GCS_USER_PROJECT 42825 - Type: string 42826 - Required: false 42827 42828 #### --gcs-service-account-file 42829 42830 Service Account Credentials JSON file path. 42831 42832 Leave blank normally. 42833 Needed only if you want use SA instead of interactive login. 42834 42835 Leading \[ga]\[ti]\[ga] will be expanded in the file name as will environment variables such as \[ga]${RCLONE_CONFIG_DIR}\[ga]. 42836 42837 Properties: 42838 42839 - Config: service_account_file 42840 - Env Var: RCLONE_GCS_SERVICE_ACCOUNT_FILE 42841 - Type: string 42842 - Required: false 42843 42844 #### --gcs-service-account-credentials 42845 42846 Service Account Credentials JSON blob. 42847 42848 Leave blank normally. 42849 Needed only if you want use SA instead of interactive login. 42850 42851 Properties: 42852 42853 - Config: service_account_credentials 42854 - Env Var: RCLONE_GCS_SERVICE_ACCOUNT_CREDENTIALS 42855 - Type: string 42856 - Required: false 42857 42858 #### --gcs-anonymous 42859 42860 Access public buckets and objects without credentials. 42861 42862 Set to \[aq]true\[aq] if you just want to download files and don\[aq]t configure credentials. 42863 42864 Properties: 42865 42866 - Config: anonymous 42867 - Env Var: RCLONE_GCS_ANONYMOUS 42868 - Type: bool 42869 - Default: false 42870 42871 #### --gcs-object-acl 42872 42873 Access Control List for new objects. 42874 42875 Properties: 42876 42877 - Config: object_acl 42878 - Env Var: RCLONE_GCS_OBJECT_ACL 42879 - Type: string 42880 - Required: false 42881 - Examples: 42882 - \[dq]authenticatedRead\[dq] 42883 - Object owner gets OWNER access. 42884 - All Authenticated Users get READER access. 42885 - \[dq]bucketOwnerFullControl\[dq] 42886 - Object owner gets OWNER access. 42887 - Project team owners get OWNER access. 42888 - \[dq]bucketOwnerRead\[dq] 42889 - Object owner gets OWNER access. 42890 - Project team owners get READER access. 42891 - \[dq]private\[dq] 42892 - Object owner gets OWNER access. 42893 - Default if left blank. 42894 - \[dq]projectPrivate\[dq] 42895 - Object owner gets OWNER access. 42896 - Project team members get access according to their roles. 42897 - \[dq]publicRead\[dq] 42898 - Object owner gets OWNER access. 42899 - All Users get READER access. 42900 42901 #### --gcs-bucket-acl 42902 42903 Access Control List for new buckets. 42904 42905 Properties: 42906 42907 - Config: bucket_acl 42908 - Env Var: RCLONE_GCS_BUCKET_ACL 42909 - Type: string 42910 - Required: false 42911 - Examples: 42912 - \[dq]authenticatedRead\[dq] 42913 - Project team owners get OWNER access. 42914 - All Authenticated Users get READER access. 42915 - \[dq]private\[dq] 42916 - Project team owners get OWNER access. 42917 - Default if left blank. 42918 - \[dq]projectPrivate\[dq] 42919 - Project team members get access according to their roles. 42920 - \[dq]publicRead\[dq] 42921 - Project team owners get OWNER access. 42922 - All Users get READER access. 42923 - \[dq]publicReadWrite\[dq] 42924 - Project team owners get OWNER access. 42925 - All Users get WRITER access. 42926 42927 #### --gcs-bucket-policy-only 42928 42929 Access checks should use bucket-level IAM policies. 42930 42931 If you want to upload objects to a bucket with Bucket Policy Only set 42932 then you will need to set this. 42933 42934 When it is set, rclone: 42935 42936 - ignores ACLs set on buckets 42937 - ignores ACLs set on objects 42938 - creates buckets with Bucket Policy Only set 42939 42940 Docs: https://cloud.google.com/storage/docs/bucket-policy-only 42941 42942 42943 Properties: 42944 42945 - Config: bucket_policy_only 42946 - Env Var: RCLONE_GCS_BUCKET_POLICY_ONLY 42947 - Type: bool 42948 - Default: false 42949 42950 #### --gcs-location 42951 42952 Location for the newly created buckets. 42953 42954 Properties: 42955 42956 - Config: location 42957 - Env Var: RCLONE_GCS_LOCATION 42958 - Type: string 42959 - Required: false 42960 - Examples: 42961 - \[dq]\[dq] 42962 - Empty for default location (US) 42963 - \[dq]asia\[dq] 42964 - Multi-regional location for Asia 42965 - \[dq]eu\[dq] 42966 - Multi-regional location for Europe 42967 - \[dq]us\[dq] 42968 - Multi-regional location for United States 42969 - \[dq]asia-east1\[dq] 42970 - Taiwan 42971 - \[dq]asia-east2\[dq] 42972 - Hong Kong 42973 - \[dq]asia-northeast1\[dq] 42974 - Tokyo 42975 - \[dq]asia-northeast2\[dq] 42976 - Osaka 42977 - \[dq]asia-northeast3\[dq] 42978 - Seoul 42979 - \[dq]asia-south1\[dq] 42980 - Mumbai 42981 - \[dq]asia-south2\[dq] 42982 - Delhi 42983 - \[dq]asia-southeast1\[dq] 42984 - Singapore 42985 - \[dq]asia-southeast2\[dq] 42986 - Jakarta 42987 - \[dq]australia-southeast1\[dq] 42988 - Sydney 42989 - \[dq]australia-southeast2\[dq] 42990 - Melbourne 42991 - \[dq]europe-north1\[dq] 42992 - Finland 42993 - \[dq]europe-west1\[dq] 42994 - Belgium 42995 - \[dq]europe-west2\[dq] 42996 - London 42997 - \[dq]europe-west3\[dq] 42998 - Frankfurt 42999 - \[dq]europe-west4\[dq] 43000 - Netherlands 43001 - \[dq]europe-west6\[dq] 43002 - Z\[:u]rich 43003 - \[dq]europe-central2\[dq] 43004 - Warsaw 43005 - \[dq]us-central1\[dq] 43006 - Iowa 43007 - \[dq]us-east1\[dq] 43008 - South Carolina 43009 - \[dq]us-east4\[dq] 43010 - Northern Virginia 43011 - \[dq]us-west1\[dq] 43012 - Oregon 43013 - \[dq]us-west2\[dq] 43014 - California 43015 - \[dq]us-west3\[dq] 43016 - Salt Lake City 43017 - \[dq]us-west4\[dq] 43018 - Las Vegas 43019 - \[dq]northamerica-northeast1\[dq] 43020 - Montr\['e]al 43021 - \[dq]northamerica-northeast2\[dq] 43022 - Toronto 43023 - \[dq]southamerica-east1\[dq] 43024 - S\[~a]o Paulo 43025 - \[dq]southamerica-west1\[dq] 43026 - Santiago 43027 - \[dq]asia1\[dq] 43028 - Dual region: asia-northeast1 and asia-northeast2. 43029 - \[dq]eur4\[dq] 43030 - Dual region: europe-north1 and europe-west4. 43031 - \[dq]nam4\[dq] 43032 - Dual region: us-central1 and us-east1. 43033 43034 #### --gcs-storage-class 43035 43036 The storage class to use when storing objects in Google Cloud Storage. 43037 43038 Properties: 43039 43040 - Config: storage_class 43041 - Env Var: RCLONE_GCS_STORAGE_CLASS 43042 - Type: string 43043 - Required: false 43044 - Examples: 43045 - \[dq]\[dq] 43046 - Default 43047 - \[dq]MULTI_REGIONAL\[dq] 43048 - Multi-regional storage class 43049 - \[dq]REGIONAL\[dq] 43050 - Regional storage class 43051 - \[dq]NEARLINE\[dq] 43052 - Nearline storage class 43053 - \[dq]COLDLINE\[dq] 43054 - Coldline storage class 43055 - \[dq]ARCHIVE\[dq] 43056 - Archive storage class 43057 - \[dq]DURABLE_REDUCED_AVAILABILITY\[dq] 43058 - Durable reduced availability storage class 43059 43060 #### --gcs-env-auth 43061 43062 Get GCP IAM credentials from runtime (environment variables or instance meta data if no env vars). 43063 43064 Only applies if service_account_file and service_account_credentials is blank. 43065 43066 Properties: 43067 43068 - Config: env_auth 43069 - Env Var: RCLONE_GCS_ENV_AUTH 43070 - Type: bool 43071 - Default: false 43072 - Examples: 43073 - \[dq]false\[dq] 43074 - Enter credentials in the next step. 43075 - \[dq]true\[dq] 43076 - Get GCP IAM credentials from the environment (env vars or IAM). 43077 43078 ### Advanced options 43079 43080 Here are the Advanced options specific to google cloud storage (Google Cloud Storage (this is not Google Drive)). 43081 43082 #### --gcs-token 43083 43084 OAuth Access Token as a JSON blob. 43085 43086 Properties: 43087 43088 - Config: token 43089 - Env Var: RCLONE_GCS_TOKEN 43090 - Type: string 43091 - Required: false 43092 43093 #### --gcs-auth-url 43094 43095 Auth server URL. 43096 43097 Leave blank to use the provider defaults. 43098 43099 Properties: 43100 43101 - Config: auth_url 43102 - Env Var: RCLONE_GCS_AUTH_URL 43103 - Type: string 43104 - Required: false 43105 43106 #### --gcs-token-url 43107 43108 Token server url. 43109 43110 Leave blank to use the provider defaults. 43111 43112 Properties: 43113 43114 - Config: token_url 43115 - Env Var: RCLONE_GCS_TOKEN_URL 43116 - Type: string 43117 - Required: false 43118 43119 #### --gcs-directory-markers 43120 43121 Upload an empty object with a trailing slash when a new directory is created 43122 43123 Empty folders are unsupported for bucket based remotes, this option creates an empty 43124 object ending with \[dq]/\[dq], to persist the folder. 43125 43126 43127 Properties: 43128 43129 - Config: directory_markers 43130 - Env Var: RCLONE_GCS_DIRECTORY_MARKERS 43131 - Type: bool 43132 - Default: false 43133 43134 #### --gcs-no-check-bucket 43135 43136 If set, don\[aq]t attempt to check the bucket exists or create it. 43137 43138 This can be useful when trying to minimise the number of transactions 43139 rclone does if you know the bucket exists already. 43140 43141 43142 Properties: 43143 43144 - Config: no_check_bucket 43145 - Env Var: RCLONE_GCS_NO_CHECK_BUCKET 43146 - Type: bool 43147 - Default: false 43148 43149 #### --gcs-decompress 43150 43151 If set this will decompress gzip encoded objects. 43152 43153 It is possible to upload objects to GCS with \[dq]Content-Encoding: gzip\[dq] 43154 set. Normally rclone will download these files as compressed objects. 43155 43156 If this flag is set then rclone will decompress these files with 43157 \[dq]Content-Encoding: gzip\[dq] as they are received. This means that rclone 43158 can\[aq]t check the size and hash but the file contents will be decompressed. 43159 43160 43161 Properties: 43162 43163 - Config: decompress 43164 - Env Var: RCLONE_GCS_DECOMPRESS 43165 - Type: bool 43166 - Default: false 43167 43168 #### --gcs-endpoint 43169 43170 Endpoint for the service. 43171 43172 Leave blank normally. 43173 43174 Properties: 43175 43176 - Config: endpoint 43177 - Env Var: RCLONE_GCS_ENDPOINT 43178 - Type: string 43179 - Required: false 43180 43181 #### --gcs-encoding 43182 43183 The encoding for the backend. 43184 43185 See the [encoding section in the overview](https://rclone.org/overview/#encoding) for more info. 43186 43187 Properties: 43188 43189 - Config: encoding 43190 - Env Var: RCLONE_GCS_ENCODING 43191 - Type: Encoding 43192 - Default: Slash,CrLf,InvalidUtf8,Dot 43193 43194 #### --gcs-description 43195 43196 Description of the remote 43197 43198 Properties: 43199 43200 - Config: description 43201 - Env Var: RCLONE_GCS_DESCRIPTION 43202 - Type: string 43203 - Required: false 43204 43205 43206 43207 ## Limitations 43208 43209 \[ga]rclone about\[ga] is not supported by the Google Cloud Storage backend. Backends without 43210 this capability cannot determine free space for an rclone mount or 43211 use policy \[ga]mfs\[ga] (most free space) as a member of an rclone union 43212 remote. 43213 43214 See [List of backends that do not support rclone about](https://rclone.org/overview/#optional-features) and [rclone about](https://rclone.org/commands/rclone_about/) 43215 43216 # Google Drive 43217 43218 Paths are specified as \[ga]drive:path\[ga] 43219 43220 Drive paths may be as deep as required, e.g. \[ga]drive:directory/subdirectory\[ga]. 43221 43222 ## Configuration 43223 43224 The initial setup for drive involves getting a token from Google drive 43225 which you need to do in your browser. \[ga]rclone config\[ga] walks you 43226 through it. 43227 43228 Here is an example of how to make a remote called \[ga]remote\[ga]. First run: 43229 43230 rclone config 43231 43232 This will guide you through an interactive setup process: 43233 \f[R] 43234 .fi 43235 .PP 43236 No remotes found, make a new one? 43237 n) New remote r) Rename remote c) Copy remote s) Set configuration 43238 password q) Quit config n/r/c/s/q> n name> remote Type of storage to 43239 configure. 43240 Choose a number from below, or type in your own value [snip] XX / Google 43241 Drive \ \[dq]drive\[dq] [snip] Storage> drive Google Application Client 43242 Id - leave blank normally. 43243 client_id> Google Application Client Secret - leave blank normally. 43244 client_secret> Scope that rclone should use when requesting access from 43245 drive. 43246 Choose a number from below, or type in your own value 1 / Full access 43247 all files, excluding Application Data Folder. 43248 \ \[dq]drive\[dq] 2 / Read-only access to file metadata and file 43249 contents. 43250 \ \[dq]drive.readonly\[dq] / Access to files created by rclone only. 43251 3 | These are visible in the drive website. 43252 | File authorization is revoked when the user deauthorizes the app. 43253 \ \[dq]drive.file\[dq] / Allows read and write access to the Application 43254 Data folder. 43255 4 | This is not visible in the drive website. 43256 \ \[dq]drive.appfolder\[dq] / Allows read-only access to file metadata 43257 but 5 | does not allow any access to read or download file content. 43258 \ \[dq]drive.metadata.readonly\[dq] scope> 1 Service Account Credentials 43259 JSON file path - needed only if you want use SA instead of interactive 43260 login. 43261 service_account_file> Remote config Use web browser to automatically 43262 authenticate rclone with remote? 43263 * Say Y if the machine running rclone has a web browser you can use * 43264 Say N if running rclone on a (remote) machine without web browser access 43265 If not sure try Y. 43266 If Y failed, try N. 43267 y) Yes n) No y/n> y If your browser doesn\[aq]t open automatically go to 43268 the following link: http://127.0.0.1:53682/auth Log in and authorize 43269 rclone for access Waiting for code... 43270 Got code Configure this as a Shared Drive (Team Drive)? 43271 y) Yes n) No y/n> n -------------------- [remote] client_id = 43272 client_secret = scope = drive root_folder_id = service_account_file = 43273 token = 43274 {\[dq]access_token\[dq]:\[dq]XXX\[dq],\[dq]token_type\[dq]:\[dq]Bearer\[dq],\[dq]refresh_token\[dq]:\[dq]XXX\[dq],\[dq]expiry\[dq]:\[dq]2014-03-16T13:57:58.955387075Z\[dq]} 43275 -------------------- y) Yes this is OK e) Edit this remote d) Delete 43276 this remote y/e/d> y 43277 .IP 43278 .nf 43279 \f[C] 43280 See the [remote setup docs](https://rclone.org/remote_setup/) for how to set it up on a 43281 machine with no Internet browser available. 43282 43283 Note that rclone runs a webserver on your local machine to collect the 43284 token as returned from Google if using web browser to automatically 43285 authenticate. This only 43286 runs from the moment it opens your browser to the moment you get back 43287 the verification code. This is on \[ga]http://127.0.0.1:53682/\[ga] and it 43288 may require you to unblock it temporarily if you are running a host 43289 firewall, or use manual mode. 43290 43291 You can then use it like this, 43292 43293 List directories in top level of your drive 43294 43295 rclone lsd remote: 43296 43297 List all the files in your drive 43298 43299 rclone ls remote: 43300 43301 To copy a local directory to a drive directory called backup 43302 43303 rclone copy /home/source remote:backup 43304 43305 ### Scopes 43306 43307 Rclone allows you to select which scope you would like for rclone to 43308 use. This changes what type of token is granted to rclone. [The 43309 scopes are defined 43310 here](https://developers.google.com/drive/v3/web/about-auth). 43311 43312 A comma-separated list is allowed e.g. \[ga]drive.readonly,drive.file\[ga]. 43313 43314 The scope are 43315 43316 #### drive 43317 43318 This is the default scope and allows full access to all files, except 43319 for the Application Data Folder (see below). 43320 43321 Choose this one if you aren\[aq]t sure. 43322 43323 #### drive.readonly 43324 43325 This allows read only access to all files. Files may be listed and 43326 downloaded but not uploaded, renamed or deleted. 43327 43328 #### drive.file 43329 43330 With this scope rclone can read/view/modify only those files and 43331 folders it creates. 43332 43333 So if you uploaded files to drive via the web interface (or any other 43334 means) they will not be visible to rclone. 43335 43336 This can be useful if you are using rclone to backup data and you want 43337 to be sure confidential data on your drive is not visible to rclone. 43338 43339 Files created with this scope are visible in the web interface. 43340 43341 #### drive.appfolder 43342 43343 This gives rclone its own private area to store files. Rclone will 43344 not be able to see any other files on your drive and you won\[aq]t be able 43345 to see rclone\[aq]s files from the web interface either. 43346 43347 #### drive.metadata.readonly 43348 43349 This allows read only access to file names only. It does not allow 43350 rclone to download or upload data, or rename or delete files or 43351 directories. 43352 43353 ### Root folder ID 43354 43355 This option has been moved to the advanced section. You can set the \[ga]root_folder_id\[ga] for rclone. This is the directory 43356 (identified by its \[ga]Folder ID\[ga]) that rclone considers to be the root 43357 of your drive. 43358 43359 Normally you will leave this blank and rclone will determine the 43360 correct root to use itself. 43361 43362 However you can set this to restrict rclone to a specific folder 43363 hierarchy or to access data within the \[dq]Computers\[dq] tab on the drive 43364 web interface (where files from Google\[aq]s Backup and Sync desktop 43365 program go). 43366 43367 In order to do this you will have to find the \[ga]Folder ID\[ga] of the 43368 directory you wish rclone to display. This will be the last segment 43369 of the URL when you open the relevant folder in the drive web 43370 interface. 43371 43372 So if the folder you want rclone to use has a URL which looks like 43373 \[ga]https://drive.google.com/drive/folders/1XyfxxxxxxxxxxxxxxxxxxxxxxxxxKHCh\[ga] 43374 in the browser, then you use \[ga]1XyfxxxxxxxxxxxxxxxxxxxxxxxxxKHCh\[ga] as 43375 the \[ga]root_folder_id\[ga] in the config. 43376 43377 **NB** folders under the \[dq]Computers\[dq] tab seem to be read only (drive 43378 gives a 500 error) when using rclone. 43379 43380 There doesn\[aq]t appear to be an API to discover the folder IDs of the 43381 \[dq]Computers\[dq] tab - please contact us if you know otherwise! 43382 43383 Note also that rclone can\[aq]t access any data under the \[dq]Backups\[dq] tab on 43384 the google drive web interface yet. 43385 43386 ### Service Account support 43387 43388 You can set up rclone with Google Drive in an unattended mode, 43389 i.e. not tied to a specific end-user Google account. This is useful 43390 when you want to synchronise files onto machines that don\[aq]t have 43391 actively logged-in users, for example build machines. 43392 43393 To use a Service Account instead of OAuth2 token flow, enter the path 43394 to your Service Account credentials at the \[ga]service_account_file\[ga] 43395 prompt during \[ga]rclone config\[ga] and rclone won\[aq]t use the browser based 43396 authentication flow. If you\[aq]d rather stuff the contents of the 43397 credentials file into the rclone config file, you can set 43398 \[ga]service_account_credentials\[ga] with the actual contents of the file 43399 instead, or set the equivalent environment variable. 43400 43401 #### Use case - Google Apps/G-suite account and individual Drive 43402 43403 Let\[aq]s say that you are the administrator of a Google Apps (old) or 43404 G-suite account. 43405 The goal is to store data on an individual\[aq]s Drive account, who IS 43406 a member of the domain. 43407 We\[aq]ll call the domain **example.com**, and the user 43408 **foo\[at]example.com**. 43409 43410 There\[aq]s a few steps we need to go through to accomplish this: 43411 43412 ##### 1. Create a service account for example.com 43413 - To create a service account and obtain its credentials, go to the 43414 [Google Developer Console](https://console.developers.google.com). 43415 - You must have a project - create one if you don\[aq]t. 43416 - Then go to \[dq]IAM & admin\[dq] -> \[dq]Service Accounts\[dq]. 43417 - Use the \[dq]Create Service Account\[dq] button. Fill in \[dq]Service account name\[dq] 43418 and \[dq]Service account ID\[dq] with something that identifies your client. 43419 - Select \[dq]Create And Continue\[dq]. Step 2 and 3 are optional. 43420 - These credentials are what rclone will use for authentication. 43421 If you ever need to remove access, press the \[dq]Delete service 43422 account key\[dq] button. 43423 43424 ##### 2. Allowing API access to example.com Google Drive 43425 - Go to example.com\[aq]s admin console 43426 - Go into \[dq]Security\[dq] (or use the search bar) 43427 - Select \[dq]Show more\[dq] and then \[dq]Advanced settings\[dq] 43428 - Select \[dq]Manage API client access\[dq] in the \[dq]Authentication\[dq] section 43429 - In the \[dq]Client Name\[dq] field enter the service account\[aq]s 43430 \[dq]Client ID\[dq] - this can be found in the Developer Console under 43431 \[dq]IAM & Admin\[dq] -> \[dq]Service Accounts\[dq], then \[dq]View Client ID\[dq] for 43432 the newly created service account. 43433 It is a \[ti]21 character numerical string. 43434 - In the next field, \[dq]One or More API Scopes\[dq], enter 43435 \[ga]https://www.googleapis.com/auth/drive\[ga] 43436 to grant access to Google Drive specifically. 43437 43438 ##### 3. Configure rclone, assuming a new install 43439 \f[R] 43440 .fi 43441 .PP 43442 rclone config 43443 .PP 43444 n/s/q> n # New name>gdrive # Gdrive is an example name Storage> # Select 43445 the number shown for Google Drive client_id> # Can be left blank 43446 client_secret> # Can be left blank scope> # Select your scope, 1 for 43447 example root_folder_id> # Can be left blank service_account_file> 43448 /home/foo/myJSONfile.json # This is where the JSON file goes! y/n> # 43449 Auto config, n 43450 .IP 43451 .nf 43452 \f[C] 43453 ##### 4. Verify that it\[aq]s working 43454 - \[ga]rclone -v --drive-impersonate foo\[at]example.com lsf gdrive:backup\[ga] 43455 - The arguments do: 43456 - \[ga]-v\[ga] - verbose logging 43457 - \[ga]--drive-impersonate foo\[at]example.com\[ga] - this is what does 43458 the magic, pretending to be user foo. 43459 - \[ga]lsf\[ga] - list files in a parsing friendly way 43460 - \[ga]gdrive:backup\[ga] - use the remote called gdrive, work in 43461 the folder named backup. 43462 43463 Note: in case you configured a specific root folder on gdrive and rclone is unable to access the contents of that folder when using \[ga]--drive-impersonate\[ga], do this instead: 43464 - in the gdrive web interface, share your root folder with the user/email of the new Service Account you created/selected at step #1 43465 - use rclone without specifying the \[ga]--drive-impersonate\[ga] option, like this: 43466 \[ga]rclone -v lsf gdrive:backup\[ga] 43467 43468 43469 ### Shared drives (team drives) 43470 43471 If you want to configure the remote to point to a Google Shared Drive 43472 (previously known as Team Drives) then answer \[ga]y\[ga] to the question 43473 \[ga]Configure this as a Shared Drive (Team Drive)?\[ga]. 43474 43475 This will fetch the list of Shared Drives from google and allow you to 43476 configure which one you want to use. You can also type in a Shared 43477 Drive ID if you prefer. 43478 43479 For example: 43480 \f[R] 43481 .fi 43482 .PP 43483 Configure this as a Shared Drive (Team Drive)? 43484 y) Yes n) No y/n> y Fetching Shared Drive list... 43485 Choose a number from below, or type in your own value 1 / Rclone Test 43486 \ \[dq]xxxxxxxxxxxxxxxxxxxx\[dq] 2 / Rclone Test 2 43487 \ \[dq]yyyyyyyyyyyyyyyyyyyy\[dq] 3 / Rclone Test 3 43488 \ \[dq]zzzzzzzzzzzzzzzzzzzz\[dq] Enter a Shared Drive ID> 1 43489 -------------------- [remote] client_id = client_secret = token = 43490 {\[dq]AccessToken\[dq]:\[dq]xxxx.x.xxxxx_xxxxxxxxxxx_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\[dq],\[dq]RefreshToken\[dq]:\[dq]1/xxxxxxxxxxxxxxxx_xxxxxxxxxxxxxxxxxxxxxxxxxx\[dq],\[dq]Expiry\[dq]:\[dq]2014-03-16T13:57:58.955387075Z\[dq],\[dq]Extra\[dq]:null} 43491 team_drive = xxxxxxxxxxxxxxxxxxxx -------------------- y) Yes this is OK 43492 e) Edit this remote d) Delete this remote y/e/d> y 43493 .IP 43494 .nf 43495 \f[C] 43496 ### --fast-list 43497 43498 This remote supports \[ga]--fast-list\[ga] which allows you to use fewer 43499 transactions in exchange for more memory. See the [rclone 43500 docs](https://rclone.org/docs/#fast-list) for more details. 43501 43502 It does this by combining multiple \[ga]list\[ga] calls into a single API request. 43503 43504 This works by combining many \[ga]\[aq]%s\[aq] in parents\[ga] filters into one expression. 43505 To list the contents of directories a, b and c, the following requests will be send by the regular \[ga]List\[ga] function: 43506 \f[R] 43507 .fi 43508 .PP 43509 trashed=false and \[aq]a\[aq] in parents trashed=false and \[aq]b\[aq] 43510 in parents trashed=false and \[aq]c\[aq] in parents 43511 .IP 43512 .nf 43513 \f[C] 43514 These can now be combined into a single request: 43515 \f[R] 43516 .fi 43517 .PP 43518 trashed=false and (\[aq]a\[aq] in parents or \[aq]b\[aq] in parents or 43519 \[aq]c\[aq] in parents) 43520 .IP 43521 .nf 43522 \f[C] 43523 The implementation of \[ga]ListR\[ga] will put up to 50 \[ga]parents\[ga] filters into one request. 43524 It will use the \[ga]--checkers\[ga] value to specify the number of requests to run in parallel. 43525 43526 In tests, these batch requests were up to 20x faster than the regular method. 43527 Running the following command against different sized folders gives: 43528 \f[R] 43529 .fi 43530 .PP 43531 rclone lsjson -vv -R --checkers=6 gdrive:folder 43532 .IP 43533 .nf 43534 \f[C] 43535 small folder (220 directories, 700 files): 43536 43537 - without \[ga]--fast-list\[ga]: 38s 43538 - with \[ga]--fast-list\[ga]: 10s 43539 43540 large folder (10600 directories, 39000 files): 43541 43542 - without \[ga]--fast-list\[ga]: 22:05 min 43543 - with \[ga]--fast-list\[ga]: 58s 43544 43545 ### Modification times and hashes 43546 43547 Google drive stores modification times accurate to 1 ms. 43548 43549 Hash algorithms MD5, SHA1 and SHA256 are supported. Note, however, 43550 that a small fraction of files uploaded may not have SHA1 or SHA256 43551 hashes especially if they were uploaded before 2018. 43552 43553 ### Restricted filename characters 43554 43555 Only Invalid UTF-8 bytes will be [replaced](https://rclone.org/overview/#invalid-utf8), 43556 as they can\[aq]t be used in JSON strings. 43557 43558 In contrast to other backends, \[ga]/\[ga] can also be used in names and \[ga].\[ga] 43559 or \[ga]..\[ga] are valid names. 43560 43561 ### Revisions 43562 43563 Google drive stores revisions of files. When you upload a change to 43564 an existing file to google drive using rclone it will create a new 43565 revision of that file. 43566 43567 Revisions follow the standard google policy which at time of writing 43568 was 43569 43570 * They are deleted after 30 days or 100 revisions (whatever comes first). 43571 * They do not count towards a user storage quota. 43572 43573 ### Deleting files 43574 43575 By default rclone will send all files to the trash when deleting 43576 files. If deleting them permanently is required then use the 43577 \[ga]--drive-use-trash=false\[ga] flag, or set the equivalent environment 43578 variable. 43579 43580 ### Shortcuts 43581 43582 In March 2020 Google introduced a new feature in Google Drive called 43583 [drive shortcuts](https://support.google.com/drive/answer/9700156) 43584 ([API](https://developers.google.com/drive/api/v3/shortcuts)). These 43585 will (by September 2020) [replace the ability for files or folders to 43586 be in multiple folders at once](https://cloud.google.com/blog/products/g-suite/simplifying-google-drives-folder-structure-and-sharing-models). 43587 43588 Shortcuts are files that link to other files on Google Drive somewhat 43589 like a symlink in unix, except they point to the underlying file data 43590 (e.g. the inode in unix terms) so they don\[aq]t break if the source is 43591 renamed or moved about. 43592 43593 By default rclone treats these as follows. 43594 43595 For shortcuts pointing to files: 43596 43597 - When listing a file shortcut appears as the destination file. 43598 - When downloading the contents of the destination file is downloaded. 43599 - When updating shortcut file with a non shortcut file, the shortcut is removed then a new file is uploaded in place of the shortcut. 43600 - When server-side moving (renaming) the shortcut is renamed, not the destination file. 43601 - When server-side copying the shortcut is copied, not the contents of the shortcut. (unless \[ga]--drive-copy-shortcut-content\[ga] is in use in which case the contents of the shortcut gets copied). 43602 - When deleting the shortcut is deleted not the linked file. 43603 - When setting the modification time, the modification time of the linked file will be set. 43604 43605 For shortcuts pointing to folders: 43606 43607 - When listing the shortcut appears as a folder and that folder will contain the contents of the linked folder appear (including any sub folders) 43608 - When downloading the contents of the linked folder and sub contents are downloaded 43609 - When uploading to a shortcut folder the file will be placed in the linked folder 43610 - When server-side moving (renaming) the shortcut is renamed, not the destination folder 43611 - When server-side copying the contents of the linked folder is copied, not the shortcut. 43612 - When deleting with \[ga]rclone rmdir\[ga] or \[ga]rclone purge\[ga] the shortcut is deleted not the linked folder. 43613 - **NB** When deleting with \[ga]rclone remove\[ga] or \[ga]rclone mount\[ga] the contents of the linked folder will be deleted. 43614 43615 The [rclone backend](https://rclone.org/commands/rclone_backend/) command can be used to create shortcuts. 43616 43617 Shortcuts can be completely ignored with the \[ga]--drive-skip-shortcuts\[ga] flag 43618 or the corresponding \[ga]skip_shortcuts\[ga] configuration setting. 43619 43620 ### Emptying trash 43621 43622 If you wish to empty your trash you can use the \[ga]rclone cleanup remote:\[ga] 43623 command which will permanently delete all your trashed files. This command 43624 does not take any path arguments. 43625 43626 Note that Google Drive takes some time (minutes to days) to empty the 43627 trash even though the command returns within a few seconds. No output 43628 is echoed, so there will be no confirmation even using -v or -vv. 43629 43630 ### Quota information 43631 43632 To view your current quota you can use the \[ga]rclone about remote:\[ga] 43633 command which will display your usage limit (quota), the usage in Google 43634 Drive, the size of all files in the Trash and the space used by other 43635 Google services such as Gmail. This command does not take any path 43636 arguments. 43637 43638 #### Import/Export of google documents 43639 43640 Google documents can be exported from and uploaded to Google Drive. 43641 43642 When rclone downloads a Google doc it chooses a format to download 43643 depending upon the \[ga]--drive-export-formats\[ga] setting. 43644 By default the export formats are \[ga]docx,xlsx,pptx,svg\[ga] which are a 43645 sensible default for an editable document. 43646 43647 When choosing a format, rclone runs down the list provided in order 43648 and chooses the first file format the doc can be exported as from the 43649 list. If the file can\[aq]t be exported to a format on the formats list, 43650 then rclone will choose a format from the default list. 43651 43652 If you prefer an archive copy then you might use \[ga]--drive-export-formats 43653 pdf\[ga], or if you prefer openoffice/libreoffice formats you might use 43654 \[ga]--drive-export-formats ods,odt,odp\[ga]. 43655 43656 Note that rclone adds the extension to the google doc, so if it is 43657 called \[ga]My Spreadsheet\[ga] on google docs, it will be exported as \[ga]My 43658 Spreadsheet.xlsx\[ga] or \[ga]My Spreadsheet.pdf\[ga] etc. 43659 43660 When importing files into Google Drive, rclone will convert all 43661 files with an extension in \[ga]--drive-import-formats\[ga] to their 43662 associated document type. 43663 rclone will not convert any files by default, since the conversion 43664 is lossy process. 43665 43666 The conversion must result in a file with the same extension when 43667 the \[ga]--drive-export-formats\[ga] rules are applied to the uploaded document. 43668 43669 Here are some examples for allowed and prohibited conversions. 43670 43671 | export-formats | import-formats | Upload Ext | Document Ext | Allowed | 43672 | -------------- | -------------- | ---------- | ------------ | ------- | 43673 | odt | odt | odt | odt | Yes | 43674 | odt | docx,odt | odt | odt | Yes | 43675 | | docx | docx | docx | Yes | 43676 | | odt | odt | docx | No | 43677 | odt,docx | docx,odt | docx | odt | No | 43678 | docx,odt | docx,odt | docx | docx | Yes | 43679 | docx,odt | docx,odt | odt | docx | No | 43680 43681 This limitation can be disabled by specifying \[ga]--drive-allow-import-name-change\[ga]. 43682 When using this flag, rclone can convert multiple files types resulting 43683 in the same document type at once, e.g. with \[ga]--drive-import-formats docx,odt,txt\[ga], 43684 all files having these extension would result in a document represented as a docx file. 43685 This brings the additional risk of overwriting a document, if multiple files 43686 have the same stem. Many rclone operations will not handle this name change 43687 in any way. They assume an equal name when copying files and might copy the 43688 file again or delete them when the name changes. 43689 43690 Here are the possible export extensions with their corresponding mime types. 43691 Most of these can also be used for importing, but there more that are not 43692 listed here. Some of these additional ones might only be available when 43693 the operating system provides the correct MIME type entries. 43694 43695 This list can be changed by Google Drive at any time and might not 43696 represent the currently available conversions. 43697 43698 | Extension | Mime Type | Description | 43699 | --------- |-----------| ------------| 43700 | bmp | image/bmp | Windows Bitmap format | 43701 | csv | text/csv | Standard CSV format for Spreadsheets | 43702 | doc | application/msword | Classic Word file | 43703 | docx | application/vnd.openxmlformats-officedocument.wordprocessingml.document | Microsoft Office Document | 43704 | epub | application/epub+zip | E-book format | 43705 | html | text/html | An HTML Document | 43706 | jpg | image/jpeg | A JPEG Image File | 43707 | json | application/vnd.google-apps.script+json | JSON Text Format for Google Apps scripts | 43708 | odp | application/vnd.oasis.opendocument.presentation | Openoffice Presentation | 43709 | ods | application/vnd.oasis.opendocument.spreadsheet | Openoffice Spreadsheet | 43710 | ods | application/x-vnd.oasis.opendocument.spreadsheet | Openoffice Spreadsheet | 43711 | odt | application/vnd.oasis.opendocument.text | Openoffice Document | 43712 | pdf | application/pdf | Adobe PDF Format | 43713 | pjpeg | image/pjpeg | Progressive JPEG Image | 43714 | png | image/png | PNG Image Format| 43715 | pptx | application/vnd.openxmlformats-officedocument.presentationml.presentation | Microsoft Office Powerpoint | 43716 | rtf | application/rtf | Rich Text Format | 43717 | svg | image/svg+xml | Scalable Vector Graphics Format | 43718 | tsv | text/tab-separated-values | Standard TSV format for spreadsheets | 43719 | txt | text/plain | Plain Text | 43720 | wmf | application/x-msmetafile | Windows Meta File | 43721 | xls | application/vnd.ms-excel | Classic Excel file | 43722 | xlsx | application/vnd.openxmlformats-officedocument.spreadsheetml.sheet | Microsoft Office Spreadsheet | 43723 | zip | application/zip | A ZIP file of HTML, Images CSS | 43724 43725 Google documents can also be exported as link files. These files will 43726 open a browser window for the Google Docs website of that document 43727 when opened. The link file extension has to be specified as a 43728 \[ga]--drive-export-formats\[ga] parameter. They will match all available 43729 Google Documents. 43730 43731 | Extension | Description | OS Support | 43732 | --------- | ----------- | ---------- | 43733 | desktop | freedesktop.org specified desktop entry | Linux | 43734 | link.html | An HTML Document with a redirect | All | 43735 | url | INI style link file | macOS, Windows | 43736 | webloc | macOS specific XML format | macOS | 43737 43738 43739 ### Standard options 43740 43741 Here are the Standard options specific to drive (Google Drive). 43742 43743 #### --drive-client-id 43744 43745 Google Application Client Id 43746 Setting your own is recommended. 43747 See https://rclone.org/drive/#making-your-own-client-id for how to create your own. 43748 If you leave this blank, it will use an internal key which is low performance. 43749 43750 Properties: 43751 43752 - Config: client_id 43753 - Env Var: RCLONE_DRIVE_CLIENT_ID 43754 - Type: string 43755 - Required: false 43756 43757 #### --drive-client-secret 43758 43759 OAuth Client Secret. 43760 43761 Leave blank normally. 43762 43763 Properties: 43764 43765 - Config: client_secret 43766 - Env Var: RCLONE_DRIVE_CLIENT_SECRET 43767 - Type: string 43768 - Required: false 43769 43770 #### --drive-scope 43771 43772 Comma separated list of scopes that rclone should use when requesting access from drive. 43773 43774 Properties: 43775 43776 - Config: scope 43777 - Env Var: RCLONE_DRIVE_SCOPE 43778 - Type: string 43779 - Required: false 43780 - Examples: 43781 - \[dq]drive\[dq] 43782 - Full access all files, excluding Application Data Folder. 43783 - \[dq]drive.readonly\[dq] 43784 - Read-only access to file metadata and file contents. 43785 - \[dq]drive.file\[dq] 43786 - Access to files created by rclone only. 43787 - These are visible in the drive website. 43788 - File authorization is revoked when the user deauthorizes the app. 43789 - \[dq]drive.appfolder\[dq] 43790 - Allows read and write access to the Application Data folder. 43791 - This is not visible in the drive website. 43792 - \[dq]drive.metadata.readonly\[dq] 43793 - Allows read-only access to file metadata but 43794 - does not allow any access to read or download file content. 43795 43796 #### --drive-service-account-file 43797 43798 Service Account Credentials JSON file path. 43799 43800 Leave blank normally. 43801 Needed only if you want use SA instead of interactive login. 43802 43803 Leading \[ga]\[ti]\[ga] will be expanded in the file name as will environment variables such as \[ga]${RCLONE_CONFIG_DIR}\[ga]. 43804 43805 Properties: 43806 43807 - Config: service_account_file 43808 - Env Var: RCLONE_DRIVE_SERVICE_ACCOUNT_FILE 43809 - Type: string 43810 - Required: false 43811 43812 #### --drive-alternate-export 43813 43814 Deprecated: No longer needed. 43815 43816 Properties: 43817 43818 - Config: alternate_export 43819 - Env Var: RCLONE_DRIVE_ALTERNATE_EXPORT 43820 - Type: bool 43821 - Default: false 43822 43823 ### Advanced options 43824 43825 Here are the Advanced options specific to drive (Google Drive). 43826 43827 #### --drive-token 43828 43829 OAuth Access Token as a JSON blob. 43830 43831 Properties: 43832 43833 - Config: token 43834 - Env Var: RCLONE_DRIVE_TOKEN 43835 - Type: string 43836 - Required: false 43837 43838 #### --drive-auth-url 43839 43840 Auth server URL. 43841 43842 Leave blank to use the provider defaults. 43843 43844 Properties: 43845 43846 - Config: auth_url 43847 - Env Var: RCLONE_DRIVE_AUTH_URL 43848 - Type: string 43849 - Required: false 43850 43851 #### --drive-token-url 43852 43853 Token server url. 43854 43855 Leave blank to use the provider defaults. 43856 43857 Properties: 43858 43859 - Config: token_url 43860 - Env Var: RCLONE_DRIVE_TOKEN_URL 43861 - Type: string 43862 - Required: false 43863 43864 #### --drive-root-folder-id 43865 43866 ID of the root folder. 43867 Leave blank normally. 43868 43869 Fill in to access \[dq]Computers\[dq] folders (see docs), or for rclone to use 43870 a non root folder as its starting point. 43871 43872 43873 Properties: 43874 43875 - Config: root_folder_id 43876 - Env Var: RCLONE_DRIVE_ROOT_FOLDER_ID 43877 - Type: string 43878 - Required: false 43879 43880 #### --drive-service-account-credentials 43881 43882 Service Account Credentials JSON blob. 43883 43884 Leave blank normally. 43885 Needed only if you want use SA instead of interactive login. 43886 43887 Properties: 43888 43889 - Config: service_account_credentials 43890 - Env Var: RCLONE_DRIVE_SERVICE_ACCOUNT_CREDENTIALS 43891 - Type: string 43892 - Required: false 43893 43894 #### --drive-team-drive 43895 43896 ID of the Shared Drive (Team Drive). 43897 43898 Properties: 43899 43900 - Config: team_drive 43901 - Env Var: RCLONE_DRIVE_TEAM_DRIVE 43902 - Type: string 43903 - Required: false 43904 43905 #### --drive-auth-owner-only 43906 43907 Only consider files owned by the authenticated user. 43908 43909 Properties: 43910 43911 - Config: auth_owner_only 43912 - Env Var: RCLONE_DRIVE_AUTH_OWNER_ONLY 43913 - Type: bool 43914 - Default: false 43915 43916 #### --drive-use-trash 43917 43918 Send files to the trash instead of deleting permanently. 43919 43920 Defaults to true, namely sending files to the trash. 43921 Use \[ga]--drive-use-trash=false\[ga] to delete files permanently instead. 43922 43923 Properties: 43924 43925 - Config: use_trash 43926 - Env Var: RCLONE_DRIVE_USE_TRASH 43927 - Type: bool 43928 - Default: true 43929 43930 #### --drive-copy-shortcut-content 43931 43932 Server side copy contents of shortcuts instead of the shortcut. 43933 43934 When doing server side copies, normally rclone will copy shortcuts as 43935 shortcuts. 43936 43937 If this flag is used then rclone will copy the contents of shortcuts 43938 rather than shortcuts themselves when doing server side copies. 43939 43940 Properties: 43941 43942 - Config: copy_shortcut_content 43943 - Env Var: RCLONE_DRIVE_COPY_SHORTCUT_CONTENT 43944 - Type: bool 43945 - Default: false 43946 43947 #### --drive-skip-gdocs 43948 43949 Skip google documents in all listings. 43950 43951 If given, gdocs practically become invisible to rclone. 43952 43953 Properties: 43954 43955 - Config: skip_gdocs 43956 - Env Var: RCLONE_DRIVE_SKIP_GDOCS 43957 - Type: bool 43958 - Default: false 43959 43960 #### --drive-show-all-gdocs 43961 43962 Show all Google Docs including non-exportable ones in listings. 43963 43964 If you try a server side copy on a Google Form without this flag, you 43965 will get this error: 43966 43967 No export formats found for \[dq]application/vnd.google-apps.form\[dq] 43968 43969 However adding this flag will allow the form to be server side copied. 43970 43971 Note that rclone doesn\[aq]t add extensions to the Google Docs file names 43972 in this mode. 43973 43974 Do **not** use this flag when trying to download Google Docs - rclone 43975 will fail to download them. 43976 43977 43978 Properties: 43979 43980 - Config: show_all_gdocs 43981 - Env Var: RCLONE_DRIVE_SHOW_ALL_GDOCS 43982 - Type: bool 43983 - Default: false 43984 43985 #### --drive-skip-checksum-gphotos 43986 43987 Skip checksums on Google photos and videos only. 43988 43989 Use this if you get checksum errors when transferring Google photos or 43990 videos. 43991 43992 Setting this flag will cause Google photos and videos to return a 43993 blank checksums. 43994 43995 Google photos are identified by being in the \[dq]photos\[dq] space. 43996 43997 Corrupted checksums are caused by Google modifying the image/video but 43998 not updating the checksum. 43999 44000 Properties: 44001 44002 - Config: skip_checksum_gphotos 44003 - Env Var: RCLONE_DRIVE_SKIP_CHECKSUM_GPHOTOS 44004 - Type: bool 44005 - Default: false 44006 44007 #### --drive-shared-with-me 44008 44009 Only show files that are shared with me. 44010 44011 Instructs rclone to operate on your \[dq]Shared with me\[dq] folder (where 44012 Google Drive lets you access the files and folders others have shared 44013 with you). 44014 44015 This works both with the \[dq]list\[dq] (lsd, lsl, etc.) and the \[dq]copy\[dq] 44016 commands (copy, sync, etc.), and with all other commands too. 44017 44018 Properties: 44019 44020 - Config: shared_with_me 44021 - Env Var: RCLONE_DRIVE_SHARED_WITH_ME 44022 - Type: bool 44023 - Default: false 44024 44025 #### --drive-trashed-only 44026 44027 Only show files that are in the trash. 44028 44029 This will show trashed files in their original directory structure. 44030 44031 Properties: 44032 44033 - Config: trashed_only 44034 - Env Var: RCLONE_DRIVE_TRASHED_ONLY 44035 - Type: bool 44036 - Default: false 44037 44038 #### --drive-starred-only 44039 44040 Only show files that are starred. 44041 44042 Properties: 44043 44044 - Config: starred_only 44045 - Env Var: RCLONE_DRIVE_STARRED_ONLY 44046 - Type: bool 44047 - Default: false 44048 44049 #### --drive-formats 44050 44051 Deprecated: See export_formats. 44052 44053 Properties: 44054 44055 - Config: formats 44056 - Env Var: RCLONE_DRIVE_FORMATS 44057 - Type: string 44058 - Required: false 44059 44060 #### --drive-export-formats 44061 44062 Comma separated list of preferred formats for downloading Google docs. 44063 44064 Properties: 44065 44066 - Config: export_formats 44067 - Env Var: RCLONE_DRIVE_EXPORT_FORMATS 44068 - Type: string 44069 - Default: \[dq]docx,xlsx,pptx,svg\[dq] 44070 44071 #### --drive-import-formats 44072 44073 Comma separated list of preferred formats for uploading Google docs. 44074 44075 Properties: 44076 44077 - Config: import_formats 44078 - Env Var: RCLONE_DRIVE_IMPORT_FORMATS 44079 - Type: string 44080 - Required: false 44081 44082 #### --drive-allow-import-name-change 44083 44084 Allow the filetype to change when uploading Google docs. 44085 44086 E.g. file.doc to file.docx. This will confuse sync and reupload every time. 44087 44088 Properties: 44089 44090 - Config: allow_import_name_change 44091 - Env Var: RCLONE_DRIVE_ALLOW_IMPORT_NAME_CHANGE 44092 - Type: bool 44093 - Default: false 44094 44095 #### --drive-use-created-date 44096 44097 Use file created date instead of modified date. 44098 44099 Useful when downloading data and you want the creation date used in 44100 place of the last modified date. 44101 44102 **WARNING**: This flag may have some unexpected consequences. 44103 44104 When uploading to your drive all files will be overwritten unless they 44105 haven\[aq]t been modified since their creation. And the inverse will occur 44106 while downloading. This side effect can be avoided by using the 44107 \[dq]--checksum\[dq] flag. 44108 44109 This feature was implemented to retain photos capture date as recorded 44110 by google photos. You will first need to check the \[dq]Create a Google 44111 Photos folder\[dq] option in your google drive settings. You can then copy 44112 or move the photos locally and use the date the image was taken 44113 (created) set as the modification date. 44114 44115 Properties: 44116 44117 - Config: use_created_date 44118 - Env Var: RCLONE_DRIVE_USE_CREATED_DATE 44119 - Type: bool 44120 - Default: false 44121 44122 #### --drive-use-shared-date 44123 44124 Use date file was shared instead of modified date. 44125 44126 Note that, as with \[dq]--drive-use-created-date\[dq], this flag may have 44127 unexpected consequences when uploading/downloading files. 44128 44129 If both this flag and \[dq]--drive-use-created-date\[dq] are set, the created 44130 date is used. 44131 44132 Properties: 44133 44134 - Config: use_shared_date 44135 - Env Var: RCLONE_DRIVE_USE_SHARED_DATE 44136 - Type: bool 44137 - Default: false 44138 44139 #### --drive-list-chunk 44140 44141 Size of listing chunk 100-1000, 0 to disable. 44142 44143 Properties: 44144 44145 - Config: list_chunk 44146 - Env Var: RCLONE_DRIVE_LIST_CHUNK 44147 - Type: int 44148 - Default: 1000 44149 44150 #### --drive-impersonate 44151 44152 Impersonate this user when using a service account. 44153 44154 Properties: 44155 44156 - Config: impersonate 44157 - Env Var: RCLONE_DRIVE_IMPERSONATE 44158 - Type: string 44159 - Required: false 44160 44161 #### --drive-upload-cutoff 44162 44163 Cutoff for switching to chunked upload. 44164 44165 Properties: 44166 44167 - Config: upload_cutoff 44168 - Env Var: RCLONE_DRIVE_UPLOAD_CUTOFF 44169 - Type: SizeSuffix 44170 - Default: 8Mi 44171 44172 #### --drive-chunk-size 44173 44174 Upload chunk size. 44175 44176 Must a power of 2 >= 256k. 44177 44178 Making this larger will improve performance, but note that each chunk 44179 is buffered in memory one per transfer. 44180 44181 Reducing this will reduce memory usage but decrease performance. 44182 44183 Properties: 44184 44185 - Config: chunk_size 44186 - Env Var: RCLONE_DRIVE_CHUNK_SIZE 44187 - Type: SizeSuffix 44188 - Default: 8Mi 44189 44190 #### --drive-acknowledge-abuse 44191 44192 Set to allow files which return cannotDownloadAbusiveFile to be downloaded. 44193 44194 If downloading a file returns the error \[dq]This file has been identified 44195 as malware or spam and cannot be downloaded\[dq] with the error code 44196 \[dq]cannotDownloadAbusiveFile\[dq] then supply this flag to rclone to 44197 indicate you acknowledge the risks of downloading the file and rclone 44198 will download it anyway. 44199 44200 Note that if you are using service account it will need Manager 44201 permission (not Content Manager) to for this flag to work. If the SA 44202 does not have the right permission, Google will just ignore the flag. 44203 44204 Properties: 44205 44206 - Config: acknowledge_abuse 44207 - Env Var: RCLONE_DRIVE_ACKNOWLEDGE_ABUSE 44208 - Type: bool 44209 - Default: false 44210 44211 #### --drive-keep-revision-forever 44212 44213 Keep new head revision of each file forever. 44214 44215 Properties: 44216 44217 - Config: keep_revision_forever 44218 - Env Var: RCLONE_DRIVE_KEEP_REVISION_FOREVER 44219 - Type: bool 44220 - Default: false 44221 44222 #### --drive-size-as-quota 44223 44224 Show sizes as storage quota usage, not actual size. 44225 44226 Show the size of a file as the storage quota used. This is the 44227 current version plus any older versions that have been set to keep 44228 forever. 44229 44230 **WARNING**: This flag may have some unexpected consequences. 44231 44232 It is not recommended to set this flag in your config - the 44233 recommended usage is using the flag form --drive-size-as-quota when 44234 doing rclone ls/lsl/lsf/lsjson/etc only. 44235 44236 If you do use this flag for syncing (not recommended) then you will 44237 need to use --ignore size also. 44238 44239 Properties: 44240 44241 - Config: size_as_quota 44242 - Env Var: RCLONE_DRIVE_SIZE_AS_QUOTA 44243 - Type: bool 44244 - Default: false 44245 44246 #### --drive-v2-download-min-size 44247 44248 If Object\[aq]s are greater, use drive v2 API to download. 44249 44250 Properties: 44251 44252 - Config: v2_download_min_size 44253 - Env Var: RCLONE_DRIVE_V2_DOWNLOAD_MIN_SIZE 44254 - Type: SizeSuffix 44255 - Default: off 44256 44257 #### --drive-pacer-min-sleep 44258 44259 Minimum time to sleep between API calls. 44260 44261 Properties: 44262 44263 - Config: pacer_min_sleep 44264 - Env Var: RCLONE_DRIVE_PACER_MIN_SLEEP 44265 - Type: Duration 44266 - Default: 100ms 44267 44268 #### --drive-pacer-burst 44269 44270 Number of API calls to allow without sleeping. 44271 44272 Properties: 44273 44274 - Config: pacer_burst 44275 - Env Var: RCLONE_DRIVE_PACER_BURST 44276 - Type: int 44277 - Default: 100 44278 44279 #### --drive-server-side-across-configs 44280 44281 Deprecated: use --server-side-across-configs instead. 44282 44283 Allow server-side operations (e.g. copy) to work across different drive configs. 44284 44285 This can be useful if you wish to do a server-side copy between two 44286 different Google drives. Note that this isn\[aq]t enabled by default 44287 because it isn\[aq]t easy to tell if it will work between any two 44288 configurations. 44289 44290 Properties: 44291 44292 - Config: server_side_across_configs 44293 - Env Var: RCLONE_DRIVE_SERVER_SIDE_ACROSS_CONFIGS 44294 - Type: bool 44295 - Default: false 44296 44297 #### --drive-disable-http2 44298 44299 Disable drive using http2. 44300 44301 There is currently an unsolved issue with the google drive backend and 44302 HTTP/2. HTTP/2 is therefore disabled by default for the drive backend 44303 but can be re-enabled here. When the issue is solved this flag will 44304 be removed. 44305 44306 See: https://github.com/artpar/artpar/issues/3631 44307 44308 44309 44310 Properties: 44311 44312 - Config: disable_http2 44313 - Env Var: RCLONE_DRIVE_DISABLE_HTTP2 44314 - Type: bool 44315 - Default: true 44316 44317 #### --drive-stop-on-upload-limit 44318 44319 Make upload limit errors be fatal. 44320 44321 At the time of writing it is only possible to upload 750 GiB of data to 44322 Google Drive a day (this is an undocumented limit). When this limit is 44323 reached Google Drive produces a slightly different error message. When 44324 this flag is set it causes these errors to be fatal. These will stop 44325 the in-progress sync. 44326 44327 Note that this detection is relying on error message strings which 44328 Google don\[aq]t document so it may break in the future. 44329 44330 See: https://github.com/artpar/artpar/issues/3857 44331 44332 44333 Properties: 44334 44335 - Config: stop_on_upload_limit 44336 - Env Var: RCLONE_DRIVE_STOP_ON_UPLOAD_LIMIT 44337 - Type: bool 44338 - Default: false 44339 44340 #### --drive-stop-on-download-limit 44341 44342 Make download limit errors be fatal. 44343 44344 At the time of writing it is only possible to download 10 TiB of data from 44345 Google Drive a day (this is an undocumented limit). When this limit is 44346 reached Google Drive produces a slightly different error message. When 44347 this flag is set it causes these errors to be fatal. These will stop 44348 the in-progress sync. 44349 44350 Note that this detection is relying on error message strings which 44351 Google don\[aq]t document so it may break in the future. 44352 44353 44354 Properties: 44355 44356 - Config: stop_on_download_limit 44357 - Env Var: RCLONE_DRIVE_STOP_ON_DOWNLOAD_LIMIT 44358 - Type: bool 44359 - Default: false 44360 44361 #### --drive-skip-shortcuts 44362 44363 If set skip shortcut files. 44364 44365 Normally rclone dereferences shortcut files making them appear as if 44366 they are the original file (see [the shortcuts section](#shortcuts)). 44367 If this flag is set then rclone will ignore shortcut files completely. 44368 44369 44370 Properties: 44371 44372 - Config: skip_shortcuts 44373 - Env Var: RCLONE_DRIVE_SKIP_SHORTCUTS 44374 - Type: bool 44375 - Default: false 44376 44377 #### --drive-skip-dangling-shortcuts 44378 44379 If set skip dangling shortcut files. 44380 44381 If this is set then rclone will not show any dangling shortcuts in listings. 44382 44383 44384 Properties: 44385 44386 - Config: skip_dangling_shortcuts 44387 - Env Var: RCLONE_DRIVE_SKIP_DANGLING_SHORTCUTS 44388 - Type: bool 44389 - Default: false 44390 44391 #### --drive-resource-key 44392 44393 Resource key for accessing a link-shared file. 44394 44395 If you need to access files shared with a link like this 44396 44397 https://drive.google.com/drive/folders/XXX?resourcekey=YYY&usp=sharing 44398 44399 Then you will need to use the first part \[dq]XXX\[dq] as the \[dq]root_folder_id\[dq] 44400 and the second part \[dq]YYY\[dq] as the \[dq]resource_key\[dq] otherwise you will get 44401 404 not found errors when trying to access the directory. 44402 44403 See: https://developers.google.com/drive/api/guides/resource-keys 44404 44405 This resource key requirement only applies to a subset of old files. 44406 44407 Note also that opening the folder once in the web interface (with the 44408 user you\[aq]ve authenticated rclone with) seems to be enough so that the 44409 resource key is not needed. 44410 44411 44412 Properties: 44413 44414 - Config: resource_key 44415 - Env Var: RCLONE_DRIVE_RESOURCE_KEY 44416 - Type: string 44417 - Required: false 44418 44419 #### --drive-fast-list-bug-fix 44420 44421 Work around a bug in Google Drive listing. 44422 44423 Normally rclone will work around a bug in Google Drive when using 44424 --fast-list (ListR) where the search \[dq](A in parents) or (B in 44425 parents)\[dq] returns nothing sometimes. See #3114, #4289 and 44426 https://issuetracker.google.com/issues/149522397 44427 44428 Rclone detects this by finding no items in more than one directory 44429 when listing and retries them as lists of individual directories. 44430 44431 This means that if you have a lot of empty directories rclone will end 44432 up listing them all individually and this can take many more API 44433 calls. 44434 44435 This flag allows the work-around to be disabled. This is **not** 44436 recommended in normal use - only if you have a particular case you are 44437 having trouble with like many empty directories. 44438 44439 44440 Properties: 44441 44442 - Config: fast_list_bug_fix 44443 - Env Var: RCLONE_DRIVE_FAST_LIST_BUG_FIX 44444 - Type: bool 44445 - Default: true 44446 44447 #### --drive-metadata-owner 44448 44449 Control whether owner should be read or written in metadata. 44450 44451 Owner is a standard part of the file metadata so is easy to read. But it 44452 isn\[aq]t always desirable to set the owner from the metadata. 44453 44454 Note that you can\[aq]t set the owner on Shared Drives, and that setting 44455 ownership will generate an email to the new owner (this can\[aq]t be 44456 disabled), and you can\[aq]t transfer ownership to someone outside your 44457 organization. 44458 44459 44460 Properties: 44461 44462 - Config: metadata_owner 44463 - Env Var: RCLONE_DRIVE_METADATA_OWNER 44464 - Type: Bits 44465 - Default: read 44466 - Examples: 44467 - \[dq]off\[dq] 44468 - Do not read or write the value 44469 - \[dq]read\[dq] 44470 - Read the value only 44471 - \[dq]write\[dq] 44472 - Write the value only 44473 - \[dq]read,write\[dq] 44474 - Read and Write the value. 44475 44476 #### --drive-metadata-permissions 44477 44478 Control whether permissions should be read or written in metadata. 44479 44480 Reading permissions metadata from files can be done quickly, but it 44481 isn\[aq]t always desirable to set the permissions from the metadata. 44482 44483 Note that rclone drops any inherited permissions on Shared Drives and 44484 any owner permission on My Drives as these are duplicated in the owner 44485 metadata. 44486 44487 44488 Properties: 44489 44490 - Config: metadata_permissions 44491 - Env Var: RCLONE_DRIVE_METADATA_PERMISSIONS 44492 - Type: Bits 44493 - Default: off 44494 - Examples: 44495 - \[dq]off\[dq] 44496 - Do not read or write the value 44497 - \[dq]read\[dq] 44498 - Read the value only 44499 - \[dq]write\[dq] 44500 - Write the value only 44501 - \[dq]read,write\[dq] 44502 - Read and Write the value. 44503 44504 #### --drive-metadata-labels 44505 44506 Control whether labels should be read or written in metadata. 44507 44508 Reading labels metadata from files takes an extra API transaction and 44509 will slow down listings. It isn\[aq]t always desirable to set the labels 44510 from the metadata. 44511 44512 The format of labels is documented in the drive API documentation at 44513 https://developers.google.com/drive/api/reference/rest/v3/Label - 44514 rclone just provides a JSON dump of this format. 44515 44516 When setting labels, the label and fields must already exist - rclone 44517 will not create them. This means that if you are transferring labels 44518 from two different accounts you will have to create the labels in 44519 advance and use the metadata mapper to translate the IDs between the 44520 two accounts. 44521 44522 44523 Properties: 44524 44525 - Config: metadata_labels 44526 - Env Var: RCLONE_DRIVE_METADATA_LABELS 44527 - Type: Bits 44528 - Default: off 44529 - Examples: 44530 - \[dq]off\[dq] 44531 - Do not read or write the value 44532 - \[dq]read\[dq] 44533 - Read the value only 44534 - \[dq]write\[dq] 44535 - Write the value only 44536 - \[dq]read,write\[dq] 44537 - Read and Write the value. 44538 44539 #### --drive-encoding 44540 44541 The encoding for the backend. 44542 44543 See the [encoding section in the overview](https://rclone.org/overview/#encoding) for more info. 44544 44545 Properties: 44546 44547 - Config: encoding 44548 - Env Var: RCLONE_DRIVE_ENCODING 44549 - Type: Encoding 44550 - Default: InvalidUtf8 44551 44552 #### --drive-env-auth 44553 44554 Get IAM credentials from runtime (environment variables or instance meta data if no env vars). 44555 44556 Only applies if service_account_file and service_account_credentials is blank. 44557 44558 Properties: 44559 44560 - Config: env_auth 44561 - Env Var: RCLONE_DRIVE_ENV_AUTH 44562 - Type: bool 44563 - Default: false 44564 - Examples: 44565 - \[dq]false\[dq] 44566 - Enter credentials in the next step. 44567 - \[dq]true\[dq] 44568 - Get GCP IAM credentials from the environment (env vars or IAM). 44569 44570 #### --drive-description 44571 44572 Description of the remote 44573 44574 Properties: 44575 44576 - Config: description 44577 - Env Var: RCLONE_DRIVE_DESCRIPTION 44578 - Type: string 44579 - Required: false 44580 44581 ### Metadata 44582 44583 User metadata is stored in the properties field of the drive object. 44584 44585 Metadata is supported on files and directories. 44586 44587 Here are the possible system metadata items for the drive backend. 44588 44589 | Name | Help | Type | Example | Read Only | 44590 |------|------|------|---------|-----------| 44591 | btime | Time of file birth (creation) with mS accuracy. Note that this is only writable on fresh uploads - it can\[aq]t be written for updates. | RFC 3339 | 2006-01-02T15:04:05.999Z07:00 | N | 44592 | content-type | The MIME type of the file. | string | text/plain | N | 44593 | copy-requires-writer-permission | Whether the options to copy, print, or download this file, should be disabled for readers and commenters. | boolean | true | N | 44594 | description | A short description of the file. | string | Contract for signing | N | 44595 | folder-color-rgb | The color for a folder or a shortcut to a folder as an RGB hex string. | string | 881133 | N | 44596 | labels | Labels attached to this file in a JSON dump of Googled drive format. Enable with --drive-metadata-labels. | JSON | [] | N | 44597 | mtime | Time of last modification with mS accuracy. | RFC 3339 | 2006-01-02T15:04:05.999Z07:00 | N | 44598 | owner | The owner of the file. Usually an email address. Enable with --drive-metadata-owner. | string | user\[at]example.com | N | 44599 | permissions | Permissions in a JSON dump of Google drive format. On shared drives these will only be present if they aren\[aq]t inherited. Enable with --drive-metadata-permissions. | JSON | {} | N | 44600 | starred | Whether the user has starred the file. | boolean | false | N | 44601 | viewed-by-me | Whether the file has been viewed by this user. | boolean | true | **Y** | 44602 | writers-can-share | Whether users with only writer permission can modify the file\[aq]s permissions. Not populated for items in shared drives. | boolean | false | N | 44603 44604 See the [metadata](https://rclone.org/docs/#metadata) docs for more info. 44605 44606 ## Backend commands 44607 44608 Here are the commands specific to the drive backend. 44609 44610 Run them with 44611 44612 rclone backend COMMAND remote: 44613 44614 The help below will explain what arguments each command takes. 44615 44616 See the [backend](https://rclone.org/commands/rclone_backend/) command for more 44617 info on how to pass options and arguments. 44618 44619 These can be run on a running backend using the rc command 44620 [backend/command](https://rclone.org/rc/#backend-command). 44621 44622 ### get 44623 44624 Get command for fetching the drive config parameters 44625 44626 rclone backend get remote: [options] [<arguments>+] 44627 44628 This is a get command which will be used to fetch the various drive config parameters 44629 44630 Usage Examples: 44631 44632 rclone backend get drive: [-o service_account_file] [-o chunk_size] 44633 rclone rc backend/command command=get fs=drive: [-o service_account_file] [-o chunk_size] 44634 44635 44636 Options: 44637 44638 - \[dq]chunk_size\[dq]: show the current upload chunk size 44639 - \[dq]service_account_file\[dq]: show the current service account file 44640 44641 ### set 44642 44643 Set command for updating the drive config parameters 44644 44645 rclone backend set remote: [options] [<arguments>+] 44646 44647 This is a set command which will be used to update the various drive config parameters 44648 44649 Usage Examples: 44650 44651 rclone backend set drive: [-o service_account_file=sa.json] [-o chunk_size=67108864] 44652 rclone rc backend/command command=set fs=drive: [-o service_account_file=sa.json] [-o chunk_size=67108864] 44653 44654 44655 Options: 44656 44657 - \[dq]chunk_size\[dq]: update the current upload chunk size 44658 - \[dq]service_account_file\[dq]: update the current service account file 44659 44660 ### shortcut 44661 44662 Create shortcuts from files or directories 44663 44664 rclone backend shortcut remote: [options] [<arguments>+] 44665 44666 This command creates shortcuts from files or directories. 44667 44668 Usage: 44669 44670 rclone backend shortcut drive: source_item destination_shortcut 44671 rclone backend shortcut drive: source_item -o target=drive2: destination_shortcut 44672 44673 In the first example this creates a shortcut from the \[dq]source_item\[dq] 44674 which can be a file or a directory to the \[dq]destination_shortcut\[dq]. The 44675 \[dq]source_item\[dq] and the \[dq]destination_shortcut\[dq] should be relative paths 44676 from \[dq]drive:\[dq] 44677 44678 In the second example this creates a shortcut from the \[dq]source_item\[dq] 44679 relative to \[dq]drive:\[dq] to the \[dq]destination_shortcut\[dq] relative to 44680 \[dq]drive2:\[dq]. This may fail with a permission error if the user 44681 authenticated with \[dq]drive2:\[dq] can\[aq]t read files from \[dq]drive:\[dq]. 44682 44683 44684 Options: 44685 44686 - \[dq]target\[dq]: optional target remote for the shortcut destination 44687 44688 ### drives 44689 44690 List the Shared Drives available to this account 44691 44692 rclone backend drives remote: [options] [<arguments>+] 44693 44694 This command lists the Shared Drives (Team Drives) available to this 44695 account. 44696 44697 Usage: 44698 44699 rclone backend [-o config] drives drive: 44700 44701 This will return a JSON list of objects like this 44702 44703 [ 44704 { 44705 \[dq]id\[dq]: \[dq]0ABCDEF-01234567890\[dq], 44706 \[dq]kind\[dq]: \[dq]drive#teamDrive\[dq], 44707 \[dq]name\[dq]: \[dq]My Drive\[dq] 44708 }, 44709 { 44710 \[dq]id\[dq]: \[dq]0ABCDEFabcdefghijkl\[dq], 44711 \[dq]kind\[dq]: \[dq]drive#teamDrive\[dq], 44712 \[dq]name\[dq]: \[dq]Test Drive\[dq] 44713 } 44714 ] 44715 44716 With the -o config parameter it will output the list in a format 44717 suitable for adding to a config file to make aliases for all the 44718 drives found and a combined drive. 44719 44720 [My Drive] 44721 type = alias 44722 remote = drive,team_drive=0ABCDEF-01234567890,root_folder_id=: 44723 44724 [Test Drive] 44725 type = alias 44726 remote = drive,team_drive=0ABCDEFabcdefghijkl,root_folder_id=: 44727 44728 [AllDrives] 44729 type = combine 44730 upstreams = \[dq]My Drive=My Drive:\[dq] \[dq]Test Drive=Test Drive:\[dq] 44731 44732 Adding this to the rclone config file will cause those team drives to 44733 be accessible with the aliases shown. Any illegal characters will be 44734 substituted with \[dq]_\[dq] and duplicate names will have numbers suffixed. 44735 It will also add a remote called AllDrives which shows all the shared 44736 drives combined into one directory tree. 44737 44738 44739 ### untrash 44740 44741 Untrash files and directories 44742 44743 rclone backend untrash remote: [options] [<arguments>+] 44744 44745 This command untrashes all the files and directories in the directory 44746 passed in recursively. 44747 44748 Usage: 44749 44750 This takes an optional directory to trash which make this easier to 44751 use via the API. 44752 44753 rclone backend untrash drive:directory 44754 rclone backend --interactive untrash drive:directory subdir 44755 44756 Use the --interactive/-i or --dry-run flag to see what would be restored before restoring it. 44757 44758 Result: 44759 44760 { 44761 \[dq]Untrashed\[dq]: 17, 44762 \[dq]Errors\[dq]: 0 44763 } 44764 44765 44766 ### copyid 44767 44768 Copy files by ID 44769 44770 rclone backend copyid remote: [options] [<arguments>+] 44771 44772 This command copies files by ID 44773 44774 Usage: 44775 44776 rclone backend copyid drive: ID path 44777 rclone backend copyid drive: ID1 path1 ID2 path2 44778 44779 It copies the drive file with ID given to the path (an rclone path which 44780 will be passed internally to rclone copyto). The ID and path pairs can be 44781 repeated. 44782 44783 The path should end with a / to indicate copy the file as named to 44784 this directory. If it doesn\[aq]t end with a / then the last path 44785 component will be used as the file name. 44786 44787 If the destination is a drive backend then server-side copying will be 44788 attempted if possible. 44789 44790 Use the --interactive/-i or --dry-run flag to see what would be copied before copying. 44791 44792 44793 ### exportformats 44794 44795 Dump the export formats for debug purposes 44796 44797 rclone backend exportformats remote: [options] [<arguments>+] 44798 44799 ### importformats 44800 44801 Dump the import formats for debug purposes 44802 44803 rclone backend importformats remote: [options] [<arguments>+] 44804 44805 44806 44807 ## Limitations 44808 44809 Drive has quite a lot of rate limiting. This causes rclone to be 44810 limited to transferring about 2 files per second only. Individual 44811 files may be transferred much faster at 100s of MiB/s but lots of 44812 small files can take a long time. 44813 44814 Server side copies are also subject to a separate rate limit. If you 44815 see User rate limit exceeded errors, wait at least 24 hours and retry. 44816 You can disable server-side copies with \[ga]--disable copy\[ga] to download 44817 and upload the files if you prefer. 44818 44819 ### Limitations of Google Docs 44820 44821 Google docs will appear as size -1 in \[ga]rclone ls\[ga], \[ga]rclone ncdu\[ga] etc, 44822 and as size 0 in anything which uses the VFS layer, e.g. \[ga]rclone mount\[ga] 44823 and \[ga]rclone serve\[ga]. When calculating directory totals, e.g. in 44824 \[ga]rclone size\[ga] and \[ga]rclone ncdu\[ga], they will be counted in as empty 44825 files. 44826 44827 This is because rclone can\[aq]t find out the size of the Google docs 44828 without downloading them. 44829 44830 Google docs will transfer correctly with \[ga]rclone sync\[ga], \[ga]rclone copy\[ga] 44831 etc as rclone knows to ignore the size when doing the transfer. 44832 44833 However an unfortunate consequence of this is that you may not be able 44834 to download Google docs using \[ga]rclone mount\[ga]. If it doesn\[aq]t work you 44835 will get a 0 sized file. If you try again the doc may gain its 44836 correct size and be downloadable. Whether it will work on not depends 44837 on the application accessing the mount and the OS you are running - 44838 experiment to find out if it does work for you! 44839 44840 ### Duplicated files 44841 44842 Sometimes, for no reason I\[aq]ve been able to track down, drive will 44843 duplicate a file that rclone uploads. Drive unlike all the other 44844 remotes can have duplicated files. 44845 44846 Duplicated files cause problems with the syncing and you will see 44847 messages in the log about duplicates. 44848 44849 Use \[ga]rclone dedupe\[ga] to fix duplicated files. 44850 44851 Note that this isn\[aq]t just a problem with rclone, even Google Photos on 44852 Android duplicates files on drive sometimes. 44853 44854 ### Rclone appears to be re-copying files it shouldn\[aq]t 44855 44856 The most likely cause of this is the duplicated file issue above - run 44857 \[ga]rclone dedupe\[ga] and check your logs for duplicate object or directory 44858 messages. 44859 44860 This can also be caused by a delay/caching on google drive\[aq]s end when 44861 comparing directory listings. Specifically with team drives used in 44862 combination with --fast-list. Files that were uploaded recently may 44863 not appear on the directory list sent to rclone when using --fast-list. 44864 44865 Waiting a moderate period of time between attempts (estimated to be 44866 approximately 1 hour) and/or not using --fast-list both seem to be 44867 effective in preventing the problem. 44868 44869 ### SHA1 or SHA256 hashes may be missing 44870 44871 All files have MD5 hashes, but a small fraction of files uploaded may 44872 not have SHA1 or SHA256 hashes especially if they were uploaded before 2018. 44873 44874 ## Making your own client_id 44875 44876 When you use rclone with Google drive in its default configuration you 44877 are using rclone\[aq]s client_id. This is shared between all the rclone 44878 users. There is a global rate limit on the number of queries per 44879 second that each client_id can do set by Google. rclone already has a 44880 high quota and I will continue to make sure it is high enough by 44881 contacting Google. 44882 44883 It is strongly recommended to use your own client ID as the default rclone ID is heavily used. If you have multiple services running, it is recommended to use an API key for each service. The default Google quota is 10 transactions per second so it is recommended to stay under that number as if you use more than that, it will cause rclone to rate limit and make things slower. 44884 44885 Here is how to create your own Google Drive client ID for rclone: 44886 44887 1. Log into the [Google API 44888 Console](https://console.developers.google.com/) with your Google 44889 account. It doesn\[aq]t matter what Google account you use. (It need not 44890 be the same account as the Google Drive you want to access) 44891 44892 2. Select a project or create a new project. 44893 44894 3. Under \[dq]ENABLE APIS AND SERVICES\[dq] search for \[dq]Drive\[dq], and enable the 44895 \[dq]Google Drive API\[dq]. 44896 44897 4. Click \[dq]Credentials\[dq] in the left-side panel (not \[dq]Create 44898 credentials\[dq], which opens the wizard). 44899 44900 5. If you already configured an \[dq]Oauth Consent Screen\[dq], then skip 44901 to the next step; if not, click on \[dq]CONFIGURE CONSENT SCREEN\[dq] button 44902 (near the top right corner of the right panel), then select \[dq]External\[dq] 44903 and click on \[dq]CREATE\[dq]; on the next screen, enter an \[dq]Application name\[dq] 44904 (\[dq]rclone\[dq] is OK); enter \[dq]User Support Email\[dq] (your own email is OK); 44905 enter \[dq]Developer Contact Email\[dq] (your own email is OK); then click on 44906 \[dq]Save\[dq] (all other data is optional). You will also have to add some scopes, 44907 including \[ga].../auth/docs\[ga] and \[ga].../auth/drive\[ga] in order to be able to edit, 44908 create and delete files with RClone. You may also want to include the 44909 \[ga]../auth/drive.metadata.readonly\[ga] scope. After adding scopes, click 44910 \[dq]Save and continue\[dq] to add test users. Be sure to add your own account to 44911 the test users. Once you\[aq]ve added yourself as a test user and saved the 44912 changes, click again on \[dq]Credentials\[dq] on the left panel to go back to 44913 the \[dq]Credentials\[dq] screen. 44914 44915 (PS: if you are a GSuite user, you could also select \[dq]Internal\[dq] instead 44916 of \[dq]External\[dq] above, but this will restrict API use to Google Workspace 44917 users in your organisation). 44918 44919 6. Click on the \[dq]+ CREATE CREDENTIALS\[dq] button at the top of the screen, 44920 then select \[dq]OAuth client ID\[dq]. 44921 44922 7. Choose an application type of \[dq]Desktop app\[dq] and click \[dq]Create\[dq]. (the default name is fine) 44923 44924 8. It will show you a client ID and client secret. Make a note of these. 44925 44926 (If you selected \[dq]External\[dq] at Step 5 continue to Step 9. 44927 If you chose \[dq]Internal\[dq] you don\[aq]t need to publish and can skip straight to 44928 Step 10 but your destination drive must be part of the same Google Workspace.) 44929 44930 9. Go to \[dq]Oauth consent screen\[dq] and then click \[dq]PUBLISH APP\[dq] button and confirm. 44931 You will also want to add yourself as a test user. 44932 44933 10. Provide the noted client ID and client secret to rclone. 44934 44935 Be aware that, due to the \[dq]enhanced security\[dq] recently introduced by 44936 Google, you are theoretically expected to \[dq]submit your app for verification\[dq] 44937 and then wait a few weeks(!) for their response; in practice, you can go right 44938 ahead and use the client ID and client secret with rclone, the only issue will 44939 be a very scary confirmation screen shown when you connect via your browser 44940 for rclone to be able to get its token-id (but as this only happens during 44941 the remote configuration, it\[aq]s not such a big deal). Keeping the application in 44942 \[dq]Testing\[dq] will work as well, but the limitation is that any grants will expire 44943 after a week, which can be annoying to refresh constantly. If, for whatever 44944 reason, a short grant time is not a problem, then keeping the application in 44945 testing mode would also be sufficient. 44946 44947 (Thanks to \[at]balazer on github for these instructions.) 44948 44949 Sometimes, creation of an OAuth consent in Google API Console fails due to an error message 44950 \[lq]The request failed because changes to one of the field of the resource is not supported\[rq]. 44951 As a convenient workaround, the necessary Google Drive API key can be created on the 44952 [Python Quickstart](https://developers.google.com/drive/api/v3/quickstart/python) page. 44953 Just push the Enable the Drive API button to receive the Client ID and Secret. 44954 Note that it will automatically create a new project in the API Console. 44955 44956 # Google Photos 44957 44958 The rclone backend for [Google Photos](https://www.google.com/photos/about/) is 44959 a specialized backend for transferring photos and videos to and from 44960 Google Photos. 44961 44962 **NB** The Google Photos API which rclone uses has quite a few 44963 limitations, so please read the [limitations section](#limitations) 44964 carefully to make sure it is suitable for your use. 44965 44966 ## Configuration 44967 44968 The initial setup for google cloud storage involves getting a token from Google Photos 44969 which you need to do in your browser. \[ga]rclone config\[ga] walks you 44970 through it. 44971 44972 Here is an example of how to make a remote called \[ga]remote\[ga]. First run: 44973 44974 rclone config 44975 44976 This will guide you through an interactive setup process: 44977 \f[R] 44978 .fi 44979 .PP 44980 No remotes found, make a new one? 44981 n) New remote s) Set configuration password q) Quit config n/s/q> n 44982 name> remote Type of storage to configure. 44983 Enter a string value. 44984 Press Enter for the default (\[dq]\[dq]). 44985 Choose a number from below, or type in your own value [snip] XX / Google 44986 Photos \ \[dq]google photos\[dq] [snip] Storage> google photos ** See 44987 help for google photos backend at: https://rclone.org/googlephotos/ ** 44988 .PP 44989 Google Application Client Id Leave blank normally. 44990 Enter a string value. 44991 Press Enter for the default (\[dq]\[dq]). 44992 client_id> Google Application Client Secret Leave blank normally. 44993 Enter a string value. 44994 Press Enter for the default (\[dq]\[dq]). 44995 client_secret> Set to make the Google Photos backend read only. 44996 .PP 44997 If you choose read only then rclone will only request read only access 44998 to your photos, otherwise rclone will request full access. 44999 Enter a boolean value (true or false). 45000 Press Enter for the default (\[dq]false\[dq]). 45001 read_only> Edit advanced config? 45002 (y/n) y) Yes n) No y/n> n Remote config Use web browser to automatically 45003 authenticate rclone with remote? 45004 * Say Y if the machine running rclone has a web browser you can use * 45005 Say N if running rclone on a (remote) machine without web browser access 45006 If not sure try Y. 45007 If Y failed, try N. 45008 y) Yes n) No y/n> y If your browser doesn\[aq]t open automatically go to 45009 the following link: http://127.0.0.1:53682/auth Log in and authorize 45010 rclone for access Waiting for code... 45011 Got code 45012 .PP 45013 *** IMPORTANT: All media items uploaded to Google Photos with rclone *** 45014 are stored in full resolution at original quality. 45015 These uploads *** will count towards storage in your Google Account. 45016 .PP 45017 .TS 45018 tab(@); 45019 lw(20.4n). 45020 T{ 45021 [remote] type = google photos token = 45022 {\[dq]access_token\[dq]:\[dq]XXX\[dq],\[dq]token_type\[dq]:\[dq]Bearer\[dq],\[dq]refresh_token\[dq]:\[dq]XXX\[dq],\[dq]expiry\[dq]:\[dq]2019-06-28T17:38:04.644930156+01:00\[dq]} 45023 T} 45024 _ 45025 T{ 45026 y) Yes this is OK e) Edit this remote d) Delete this remote y/e/d> y 45027 \[ga]\[ga]\[ga] 45028 T} 45029 T{ 45030 See the remote setup docs (https://rclone.org/remote_setup/) for how to 45031 set it up on a machine with no Internet browser available. 45032 T} 45033 T{ 45034 Note that rclone runs a webserver on your local machine to collect the 45035 token as returned from Google if using web browser to automatically 45036 authenticate. 45037 This only runs from the moment it opens your browser to the moment you 45038 get back the verification code. 45039 This is on \f[C]http://127.0.0.1:53682/\f[R] and this may require you to 45040 unblock it temporarily if you are running a host firewall, or use manual 45041 mode. 45042 T} 45043 T{ 45044 This remote is called \f[C]remote\f[R] and can now be used like this 45045 T} 45046 T{ 45047 See all the albums in your photos 45048 T} 45049 T{ 45050 rclone lsd remote:album 45051 T} 45052 T{ 45053 Make a new album 45054 T} 45055 T{ 45056 rclone mkdir remote:album/newAlbum 45057 T} 45058 T{ 45059 List the contents of an album 45060 T} 45061 T{ 45062 rclone ls remote:album/newAlbum 45063 T} 45064 T{ 45065 Sync \f[C]/home/local/images\f[R] to the Google Photos, removing any 45066 excess files in the album. 45067 T} 45068 T{ 45069 rclone sync --interactive /home/local/image remote:album/newAlbum 45070 T} 45071 T{ 45072 ### Layout 45073 T} 45074 T{ 45075 As Google Photos is not a general purpose cloud storage system, the 45076 backend is laid out to help you navigate it. 45077 T} 45078 T{ 45079 The directories under \f[C]media\f[R] show different ways of 45080 categorizing the media. 45081 Each file will appear multiple times. 45082 So if you want to make a backup of your google photos you might choose 45083 to backup \f[C]remote:media/by-month\f[R]. 45084 (\f[B]NB\f[R] \f[C]remote:media/by-day\f[R] is rather slow at the moment 45085 so avoid for syncing.) 45086 T} 45087 T{ 45088 Note that all your photos and videos will appear somewhere under 45089 \f[C]media\f[R], but they may not appear under \f[C]album\f[R] unless 45090 you\[aq]ve put them into albums. 45091 T} 45092 T{ 45093 \f[C]/ - upload - file1.jpg - file2.jpg - ... - media - all - file1.jpg - file2.jpg - ... - by-year - 2000 - file1.jpg - ... - 2001 - file2.jpg - ... - ... - by-month - 2000 - 2000-01 - file1.jpg - ... - 2000-02 - file2.jpg - ... - ... - by-day - 2000 - 2000-01-01 - file1.jpg - ... - 2000-01-02 - file2.jpg - ... - ... - album - album name - album name/sub - shared-album - album name - album name/sub - feature - favorites - file1.jpg - file2.jpg\f[R] 45094 T} 45095 T{ 45096 There are two writable parts of the tree, the \f[C]upload\f[R] directory 45097 and sub directories of the \f[C]album\f[R] directory. 45098 T} 45099 T{ 45100 The \f[C]upload\f[R] directory is for uploading files you don\[aq]t want 45101 to put into albums. 45102 This will be empty to start with and will contain the files you\[aq]ve 45103 uploaded for one rclone session only, becoming empty again when you 45104 restart rclone. 45105 The use case for this would be if you have a load of files you just want 45106 to once off dump into Google Photos. 45107 For repeated syncing, uploading to \f[C]album\f[R] will work better. 45108 T} 45109 T{ 45110 Directories within the \f[C]album\f[R] directory are also writeable and 45111 you may create new directories (albums) under \f[C]album\f[R]. 45112 If you copy files with a directory hierarchy in there then rclone will 45113 create albums with the \f[C]/\f[R] character in them. 45114 For example if you do 45115 T} 45116 T{ 45117 rclone copy /path/to/images remote:album/images 45118 T} 45119 T{ 45120 and the images directory contains 45121 T} 45122 T{ 45123 \f[C]images - file1.jpg dir file2.jpg dir2 dir3 file3.jpg\f[R] 45124 T} 45125 T{ 45126 Then rclone will create the following albums with the following files in 45127 T} 45128 T{ 45129 - images - file1.jpg - images/dir - file2.jpg - images/dir2/dir3 - 45130 file3.jpg 45131 T} 45132 T{ 45133 This means that you can use the \f[C]album\f[R] path pretty much like a 45134 normal filesystem and it is a good target for repeated syncing. 45135 T} 45136 T{ 45137 The \f[C]shared-album\f[R] directory shows albums shared with you or by 45138 you. 45139 This is similar to the Sharing tab in the Google Photos web interface. 45140 T} 45141 T{ 45142 ### Standard options 45143 T} 45144 T{ 45145 Here are the Standard options specific to google photos (Google Photos). 45146 T} 45147 T{ 45148 #### --gphotos-client-id 45149 T} 45150 T{ 45151 OAuth Client Id. 45152 T} 45153 T{ 45154 Leave blank normally. 45155 T} 45156 T{ 45157 Properties: 45158 T} 45159 T{ 45160 - Config: client_id - Env Var: RCLONE_GPHOTOS_CLIENT_ID - Type: string - 45161 Required: false 45162 T} 45163 T{ 45164 #### --gphotos-client-secret 45165 T} 45166 T{ 45167 OAuth Client Secret. 45168 T} 45169 T{ 45170 Leave blank normally. 45171 T} 45172 T{ 45173 Properties: 45174 T} 45175 T{ 45176 - Config: client_secret - Env Var: RCLONE_GPHOTOS_CLIENT_SECRET - Type: 45177 string - Required: false 45178 T} 45179 T{ 45180 #### --gphotos-read-only 45181 T} 45182 T{ 45183 Set to make the Google Photos backend read only. 45184 T} 45185 T{ 45186 If you choose read only then rclone will only request read only access 45187 to your photos, otherwise rclone will request full access. 45188 T} 45189 T{ 45190 Properties: 45191 T} 45192 T{ 45193 - Config: read_only - Env Var: RCLONE_GPHOTOS_READ_ONLY - Type: bool - 45194 Default: false 45195 T} 45196 T{ 45197 ### Advanced options 45198 T} 45199 T{ 45200 Here are the Advanced options specific to google photos (Google Photos). 45201 T} 45202 T{ 45203 #### --gphotos-token 45204 T} 45205 T{ 45206 OAuth Access Token as a JSON blob. 45207 T} 45208 T{ 45209 Properties: 45210 T} 45211 T{ 45212 - Config: token - Env Var: RCLONE_GPHOTOS_TOKEN - Type: string - 45213 Required: false 45214 T} 45215 T{ 45216 #### --gphotos-auth-url 45217 T} 45218 T{ 45219 Auth server URL. 45220 T} 45221 T{ 45222 Leave blank to use the provider defaults. 45223 T} 45224 T{ 45225 Properties: 45226 T} 45227 T{ 45228 - Config: auth_url - Env Var: RCLONE_GPHOTOS_AUTH_URL - Type: string - 45229 Required: false 45230 T} 45231 T{ 45232 #### --gphotos-token-url 45233 T} 45234 T{ 45235 Token server url. 45236 T} 45237 T{ 45238 Leave blank to use the provider defaults. 45239 T} 45240 T{ 45241 Properties: 45242 T} 45243 T{ 45244 - Config: token_url - Env Var: RCLONE_GPHOTOS_TOKEN_URL - Type: string - 45245 Required: false 45246 T} 45247 T{ 45248 #### --gphotos-read-size 45249 T} 45250 T{ 45251 Set to read the size of media items. 45252 T} 45253 T{ 45254 Normally rclone does not read the size of media items since this takes 45255 another transaction. 45256 This isn\[aq]t necessary for syncing. 45257 However rclone mount needs to know the size of files in advance of 45258 reading them, so setting this flag when using rclone mount is 45259 recommended if you want to read the media. 45260 T} 45261 T{ 45262 Properties: 45263 T} 45264 T{ 45265 - Config: read_size - Env Var: RCLONE_GPHOTOS_READ_SIZE - Type: bool - 45266 Default: false 45267 T} 45268 T{ 45269 #### --gphotos-start-year 45270 T} 45271 T{ 45272 Year limits the photos to be downloaded to those which are uploaded 45273 after the given year. 45274 T} 45275 T{ 45276 Properties: 45277 T} 45278 T{ 45279 - Config: start_year - Env Var: RCLONE_GPHOTOS_START_YEAR - Type: int - 45280 Default: 2000 45281 T} 45282 T{ 45283 #### --gphotos-include-archived 45284 T} 45285 T{ 45286 Also view and download archived media. 45287 T} 45288 T{ 45289 By default, rclone does not request archived media. 45290 Thus, when syncing, archived media is not visible in directory listings 45291 or transferred. 45292 T} 45293 T{ 45294 Note that media in albums is always visible and synced, no matter their 45295 archive status. 45296 T} 45297 T{ 45298 With this flag, archived media are always visible in directory listings 45299 and transferred. 45300 T} 45301 T{ 45302 Without this flag, archived media will not be visible in directory 45303 listings and won\[aq]t be transferred. 45304 T} 45305 T{ 45306 Properties: 45307 T} 45308 T{ 45309 - Config: include_archived - Env Var: RCLONE_GPHOTOS_INCLUDE_ARCHIVED - 45310 Type: bool - Default: false 45311 T} 45312 T{ 45313 #### --gphotos-encoding 45314 T} 45315 T{ 45316 The encoding for the backend. 45317 T} 45318 T{ 45319 See the encoding section in the 45320 overview (https://rclone.org/overview/#encoding) for more info. 45321 T} 45322 T{ 45323 Properties: 45324 T} 45325 T{ 45326 - Config: encoding - Env Var: RCLONE_GPHOTOS_ENCODING - Type: Encoding - 45327 Default: Slash,CrLf,InvalidUtf8,Dot 45328 T} 45329 T{ 45330 #### --gphotos-batch-mode 45331 T} 45332 T{ 45333 Upload file batching sync|async|off. 45334 T} 45335 T{ 45336 This sets the batch mode used by rclone. 45337 T} 45338 T{ 45339 This has 3 possible values 45340 T} 45341 T{ 45342 - off - no batching - sync - batch uploads and check completion 45343 (default) - async - batch upload and don\[aq]t check completion 45344 T} 45345 T{ 45346 Rclone will close any outstanding batches when it exits which may make a 45347 delay on quit. 45348 T} 45349 T{ 45350 Properties: 45351 T} 45352 T{ 45353 - Config: batch_mode - Env Var: RCLONE_GPHOTOS_BATCH_MODE - Type: string 45354 - Default: \[dq]sync\[dq] 45355 T} 45356 T{ 45357 #### --gphotos-batch-size 45358 T} 45359 T{ 45360 Max number of files in upload batch. 45361 T} 45362 T{ 45363 This sets the batch size of files to upload. 45364 It has to be less than 50. 45365 T} 45366 T{ 45367 By default this is 0 which means rclone which calculate the batch size 45368 depending on the setting of batch_mode. 45369 T} 45370 T{ 45371 - batch_mode: async - default batch_size is 50 - batch_mode: sync - 45372 default batch_size is the same as --transfers - batch_mode: off - not in 45373 use 45374 T} 45375 T{ 45376 Rclone will close any outstanding batches when it exits which may make a 45377 delay on quit. 45378 T} 45379 T{ 45380 Setting this is a great idea if you are uploading lots of small files as 45381 it will make them a lot quicker. 45382 You can use --transfers 32 to maximise throughput. 45383 T} 45384 T{ 45385 Properties: 45386 T} 45387 T{ 45388 - Config: batch_size - Env Var: RCLONE_GPHOTOS_BATCH_SIZE - Type: int - 45389 Default: 0 45390 T} 45391 T{ 45392 #### --gphotos-batch-timeout 45393 T} 45394 T{ 45395 Max time to allow an idle upload batch before uploading. 45396 T} 45397 T{ 45398 If an upload batch is idle for more than this long then it will be 45399 uploaded. 45400 T} 45401 T{ 45402 The default for this is 0 which means rclone will choose a sensible 45403 default based on the batch_mode in use. 45404 T} 45405 T{ 45406 - batch_mode: async - default batch_timeout is 10s - batch_mode: sync - 45407 default batch_timeout is 1s - batch_mode: off - not in use 45408 T} 45409 T{ 45410 Properties: 45411 T} 45412 T{ 45413 - Config: batch_timeout - Env Var: RCLONE_GPHOTOS_BATCH_TIMEOUT - Type: 45414 Duration - Default: 0s 45415 T} 45416 T{ 45417 #### --gphotos-batch-commit-timeout 45418 T} 45419 T{ 45420 Max time to wait for a batch to finish committing 45421 T} 45422 T{ 45423 Properties: 45424 T} 45425 T{ 45426 - Config: batch_commit_timeout - Env Var: 45427 RCLONE_GPHOTOS_BATCH_COMMIT_TIMEOUT - Type: Duration - Default: 10m0s 45428 T} 45429 T{ 45430 #### --gphotos-description 45431 T} 45432 T{ 45433 Description of the remote 45434 T} 45435 T{ 45436 Properties: 45437 T} 45438 T{ 45439 - Config: description - Env Var: RCLONE_GPHOTOS_DESCRIPTION - Type: 45440 string - Required: false 45441 T} 45442 T{ 45443 ## Limitations 45444 T} 45445 T{ 45446 Only images and videos can be uploaded. 45447 If you attempt to upload non videos or images or formats that Google 45448 Photos doesn\[aq]t understand, rclone will upload the file, then Google 45449 Photos will give an error when it is put turned into a media item. 45450 T} 45451 T{ 45452 Note that all media items uploaded to Google Photos through the API are 45453 stored in full resolution at \[dq]original quality\[dq] and 45454 \f[B]will\f[R] count towards your storage quota in your Google Account. 45455 The API does \f[B]not\f[R] offer a way to upload in \[dq]high 45456 quality\[dq] mode.. 45457 T} 45458 T{ 45459 \f[C]rclone about\f[R] is not supported by the Google Photos backend. 45460 Backends without this capability cannot determine free space for an 45461 rclone mount or use policy \f[C]mfs\f[R] (most free space) as a member 45462 of an rclone union remote. 45463 T} 45464 T{ 45465 See List of backends that do not support rclone 45466 about (https://rclone.org/overview/#optional-features) See rclone 45467 about (https://rclone.org/commands/rclone_about/) 45468 T} 45469 T{ 45470 ### Downloading Images 45471 T} 45472 T{ 45473 When Images are downloaded this strips EXIF location (according to the 45474 docs and my tests). 45475 This is a limitation of the Google Photos API and is covered by bug 45476 #112096115 (https://issuetracker.google.com/issues/112096115). 45477 T} 45478 T{ 45479 \f[B]The current google API does not allow photos to be downloaded at 45480 original resolution. This is very important if you are, for example, 45481 relying on \[dq]Google Photos\[dq] as a backup of your photos. You will 45482 not be able to use rclone to redownload original images. You could use 45483 \[aq]google takeout\[aq] to recover the original photos as a last 45484 resort\f[R] 45485 T} 45486 T{ 45487 ### Downloading Videos 45488 T} 45489 T{ 45490 When videos are downloaded they are downloaded in a really compressed 45491 version of the video compared to downloading it via the Google Photos 45492 web interface. 45493 This is covered by bug 45494 #113672044 (https://issuetracker.google.com/issues/113672044). 45495 T} 45496 T{ 45497 ### Duplicates 45498 T} 45499 T{ 45500 If a file name is duplicated in a directory then rclone will add the 45501 file ID into its name. 45502 So two files called \f[C]file.jpg\f[R] would then appear as 45503 \f[C]file {123456}.jpg\f[R] and \f[C]file {ABCDEF}.jpg\f[R] (the actual 45504 IDs are a lot longer alas!). 45505 T} 45506 T{ 45507 If you upload the same image (with the same binary data) twice then 45508 Google Photos will deduplicate it. 45509 However it will retain the filename from the first upload which may 45510 confuse rclone. 45511 For example if you uploaded an image to \f[C]upload\f[R] then uploaded 45512 the same image to \f[C]album/my_album\f[R] the filename of the image in 45513 \f[C]album/my_album\f[R] will be what it was uploaded with initially, 45514 not what you uploaded it with to \f[C]album\f[R]. 45515 In practise this shouldn\[aq]t cause too many problems. 45516 T} 45517 T{ 45518 ### Modification times 45519 T} 45520 T{ 45521 The date shown of media in Google Photos is the creation date as 45522 determined by the EXIF information, or the upload date if that is not 45523 known. 45524 T} 45525 T{ 45526 This is not changeable by rclone and is not the modification date of the 45527 media on local disk. 45528 This means that rclone cannot use the dates from Google Photos for 45529 syncing purposes. 45530 T} 45531 T{ 45532 ### Size 45533 T} 45534 T{ 45535 The Google Photos API does not return the size of media. 45536 This means that when syncing to Google Photos, rclone can only do a file 45537 existence check. 45538 T} 45539 T{ 45540 It is possible to read the size of the media, but this needs an extra 45541 HTTP HEAD request per media item so is \f[B]very slow\f[R] and uses up a 45542 lot of transactions. 45543 This can be enabled with the \f[C]--gphotos-read-size\f[R] option or the 45544 \f[C]read_size = true\f[R] config parameter. 45545 T} 45546 T{ 45547 If you want to use the backend with \f[C]rclone mount\f[R] you may need 45548 to enable this flag (depending on your OS and application using the 45549 photos) otherwise you may not be able to read media off the mount. 45550 You\[aq]ll need to experiment to see if it works for you without the 45551 flag. 45552 T} 45553 T{ 45554 ### Albums 45555 T} 45556 T{ 45557 Rclone can only upload files to albums it created. 45558 This is a limitation of the Google Photos 45559 API (https://developers.google.com/photos/library/guides/manage-albums). 45560 T} 45561 T{ 45562 Rclone can remove files it uploaded from albums it created only. 45563 T} 45564 T{ 45565 ### Deleting files 45566 T} 45567 T{ 45568 Rclone can remove files from albums it created, but note that the Google 45569 Photos API does not allow media to be deleted permanently so this media 45570 will still remain. 45571 See bug #109759781 (https://issuetracker.google.com/issues/109759781). 45572 T} 45573 T{ 45574 Rclone cannot delete files anywhere except under \f[C]album\f[R]. 45575 T} 45576 T{ 45577 ### Deleting albums 45578 T} 45579 T{ 45580 The Google Photos API does not support deleting albums - see bug 45581 #135714733 (https://issuetracker.google.com/issues/135714733). 45582 T} 45583 T{ 45584 # Hasher 45585 T} 45586 T{ 45587 Hasher is a special overlay backend to create remotes which handle 45588 checksums for other remotes. 45589 It\[aq]s main functions include: - Emulate hash types unimplemented by 45590 backends - Cache checksums to help with slow hashing of large local or 45591 (S)FTP files - Warm up checksum cache from external SUM files 45592 T} 45593 T{ 45594 ## Getting started 45595 T} 45596 T{ 45597 To use Hasher, first set up the underlying remote following the 45598 configuration instructions for that remote. 45599 You can also use a local pathname instead of a remote. 45600 Check that your base remote is working. 45601 T} 45602 T{ 45603 Let\[aq]s call the base remote \f[C]myRemote:path\f[R] here. 45604 Note that anything inside \f[C]myRemote:path\f[R] will be handled by 45605 hasher and anything outside won\[aq]t. 45606 This means that if you are using a bucket based remote (S3, B2, Swift) 45607 then you should put the bucket in the remote \f[C]s3:bucket\f[R]. 45608 T} 45609 T{ 45610 Now proceed to interactive or manual configuration. 45611 T} 45612 T{ 45613 ### Interactive configuration 45614 T} 45615 T{ 45616 Run \f[C]rclone config\f[R]: \[ga]\[ga]\[ga] No remotes found, make a 45617 new one? 45618 n) New remote s) Set configuration password q) Quit config n/s/q> n 45619 name> Hasher1 Type of storage to configure. 45620 Choose a number from below, or type in your own value [snip] XX / Handle 45621 checksums for other remotes \ \[dq]hasher\[dq] [snip] Storage> hasher 45622 Remote to cache checksums for, like myremote:mypath. 45623 Enter a string value. 45624 Press Enter for the default (\[dq]\[dq]). 45625 remote> myRemote:path Comma separated list of supported checksum types. 45626 Enter a string value. 45627 Press Enter for the default (\[dq]md5,sha1\[dq]). 45628 hashsums> md5 Maximum time to keep checksums in cache. 45629 0 = no cache, off = cache forever. 45630 max_age> off Edit advanced config? 45631 (y/n) y) Yes n) No y/n> n Remote config 45632 T} 45633 .TE 45634 .PP 45635 [Hasher1] type = hasher remote = myRemote:path hashsums = md5 max_age = 45636 off -------------------- y) Yes this is OK e) Edit this remote d) Delete 45637 this remote y/e/d> y 45638 .IP 45639 .nf 45640 \f[C] 45641 ### Manual configuration 45642 45643 Run \[ga]rclone config path\[ga] to see the path of current active config file, 45644 usually \[ga]YOURHOME/.config/artpar/artpar.conf\[ga]. 45645 Open it in your favorite text editor, find section for the base remote 45646 and create new section for hasher like in the following examples: 45647 \f[R] 45648 .fi 45649 .PP 45650 [Hasher1] type = hasher remote = myRemote:path hashes = md5 max_age = 45651 off 45652 .PP 45653 [Hasher2] type = hasher remote = /local/path hashes = dropbox,sha1 45654 max_age = 24h 45655 .IP 45656 .nf 45657 \f[C] 45658 Hasher takes basically the following parameters: 45659 - \[ga]remote\[ga] is required, 45660 - \[ga]hashes\[ga] is a comma separated list of supported checksums 45661 (by default \[ga]md5,sha1\[ga]), 45662 - \[ga]max_age\[ga] - maximum time to keep a checksum value in the cache, 45663 \[ga]0\[ga] will disable caching completely, 45664 \[ga]off\[ga] will cache \[dq]forever\[dq] (that is until the files get changed). 45665 45666 Make sure the \[ga]remote\[ga] has \[ga]:\[ga] (colon) in. If you specify the remote without 45667 a colon then rclone will use a local directory of that name. So if you use 45668 a remote of \[ga]/local/path\[ga] then rclone will handle hashes for that directory. 45669 If you use \[ga]remote = name\[ga] literally then rclone will put files 45670 **in a directory called \[ga]name\[ga] located under current directory**. 45671 45672 ## Usage 45673 45674 ### Basic operations 45675 45676 Now you can use it as \[ga]Hasher2:subdir/file\[ga] instead of base remote. 45677 Hasher will transparently update cache with new checksums when a file 45678 is fully read or overwritten, like: 45679 \f[R] 45680 .fi 45681 .PP 45682 rclone copy External:path/file Hasher:dest/path 45683 .PP 45684 rclone cat Hasher:path/to/file > /dev/null 45685 .IP 45686 .nf 45687 \f[C] 45688 The way to refresh **all** cached checksums (even unsupported by the base backend) 45689 for a subtree is to **re-download** all files in the subtree. For example, 45690 use \[ga]hashsum --download\[ga] using **any** supported hashsum on the command line 45691 (we just care to re-read): 45692 \f[R] 45693 .fi 45694 .PP 45695 rclone hashsum MD5 --download Hasher:path/to/subtree > /dev/null 45696 .PP 45697 rclone backend dump Hasher:path/to/subtree 45698 .IP 45699 .nf 45700 \f[C] 45701 You can print or drop hashsum cache using custom backend commands: 45702 \f[R] 45703 .fi 45704 .PP 45705 rclone backend dump Hasher:dir/subdir 45706 .PP 45707 rclone backend drop Hasher: 45708 .IP 45709 .nf 45710 \f[C] 45711 ### Pre-Seed from a SUM File 45712 45713 Hasher supports two backend commands: generic SUM file \[ga]import\[ga] and faster 45714 but less consistent \[ga]stickyimport\[ga]. 45715 \f[R] 45716 .fi 45717 .PP 45718 rclone backend import Hasher:dir/subdir SHA1 /path/to/SHA1SUM 45719 [--checkers 4] 45720 .IP 45721 .nf 45722 \f[C] 45723 Instead of SHA1 it can be any hash supported by the remote. The last argument 45724 can point to either a local or an \[ga]other-remote:path\[ga] text file in SUM format. 45725 The command will parse the SUM file, then walk down the path given by the 45726 first argument, snapshot current fingerprints and fill in the cache entries 45727 correspondingly. 45728 - Paths in the SUM file are treated as relative to \[ga]hasher:dir/subdir\[ga]. 45729 - The command will **not** check that supplied values are correct. 45730 You **must know** what you are doing. 45731 - This is a one-time action. The SUM file will not get \[dq]attached\[dq] to the 45732 remote. Cache entries can still be overwritten later, should the object\[aq]s 45733 fingerprint change. 45734 - The tree walk can take long depending on the tree size. You can increase 45735 \[ga]--checkers\[ga] to make it faster. Or use \[ga]stickyimport\[ga] if you don\[aq]t care 45736 about fingerprints and consistency. 45737 \f[R] 45738 .fi 45739 .PP 45740 rclone backend stickyimport hasher:path/to/data sha1 45741 remote:/path/to/sum.sha1 45742 .IP 45743 .nf 45744 \f[C] 45745 \[ga]stickyimport\[ga] is similar to \[ga]import\[ga] but works much faster because it 45746 does not need to stat existing files and skips initial tree walk. 45747 Instead of binding cache entries to file fingerprints it creates _sticky_ 45748 entries bound to the file name alone ignoring size, modification time etc. 45749 Such hash entries can be replaced only by \[ga]purge\[ga], \[ga]delete\[ga], \[ga]backend drop\[ga] 45750 or by full re-read/re-write of the files. 45751 45752 ## Configuration reference 45753 45754 45755 ### Standard options 45756 45757 Here are the Standard options specific to hasher (Better checksums for other remotes). 45758 45759 #### --hasher-remote 45760 45761 Remote to cache checksums for (e.g. myRemote:path). 45762 45763 Properties: 45764 45765 - Config: remote 45766 - Env Var: RCLONE_HASHER_REMOTE 45767 - Type: string 45768 - Required: true 45769 45770 #### --hasher-hashes 45771 45772 Comma separated list of supported checksum types. 45773 45774 Properties: 45775 45776 - Config: hashes 45777 - Env Var: RCLONE_HASHER_HASHES 45778 - Type: CommaSepList 45779 - Default: md5,sha1 45780 45781 #### --hasher-max-age 45782 45783 Maximum time to keep checksums in cache (0 = no cache, off = cache forever). 45784 45785 Properties: 45786 45787 - Config: max_age 45788 - Env Var: RCLONE_HASHER_MAX_AGE 45789 - Type: Duration 45790 - Default: off 45791 45792 ### Advanced options 45793 45794 Here are the Advanced options specific to hasher (Better checksums for other remotes). 45795 45796 #### --hasher-auto-size 45797 45798 Auto-update checksum for files smaller than this size (disabled by default). 45799 45800 Properties: 45801 45802 - Config: auto_size 45803 - Env Var: RCLONE_HASHER_AUTO_SIZE 45804 - Type: SizeSuffix 45805 - Default: 0 45806 45807 #### --hasher-description 45808 45809 Description of the remote 45810 45811 Properties: 45812 45813 - Config: description 45814 - Env Var: RCLONE_HASHER_DESCRIPTION 45815 - Type: string 45816 - Required: false 45817 45818 ### Metadata 45819 45820 Any metadata supported by the underlying remote is read and written. 45821 45822 See the [metadata](https://rclone.org/docs/#metadata) docs for more info. 45823 45824 ## Backend commands 45825 45826 Here are the commands specific to the hasher backend. 45827 45828 Run them with 45829 45830 rclone backend COMMAND remote: 45831 45832 The help below will explain what arguments each command takes. 45833 45834 See the [backend](https://rclone.org/commands/rclone_backend/) command for more 45835 info on how to pass options and arguments. 45836 45837 These can be run on a running backend using the rc command 45838 [backend/command](https://rclone.org/rc/#backend-command). 45839 45840 ### drop 45841 45842 Drop cache 45843 45844 rclone backend drop remote: [options] [<arguments>+] 45845 45846 Completely drop checksum cache. 45847 Usage Example: 45848 rclone backend drop hasher: 45849 45850 45851 ### dump 45852 45853 Dump the database 45854 45855 rclone backend dump remote: [options] [<arguments>+] 45856 45857 Dump cache records covered by the current remote 45858 45859 ### fulldump 45860 45861 Full dump of the database 45862 45863 rclone backend fulldump remote: [options] [<arguments>+] 45864 45865 Dump all cache records in the database 45866 45867 ### import 45868 45869 Import a SUM file 45870 45871 rclone backend import remote: [options] [<arguments>+] 45872 45873 Amend hash cache from a SUM file and bind checksums to files by size/time. 45874 Usage Example: 45875 rclone backend import hasher:subdir md5 /path/to/sum.md5 45876 45877 45878 ### stickyimport 45879 45880 Perform fast import of a SUM file 45881 45882 rclone backend stickyimport remote: [options] [<arguments>+] 45883 45884 Fill hash cache from a SUM file without verifying file fingerprints. 45885 Usage Example: 45886 rclone backend stickyimport hasher:subdir md5 remote:path/to/sum.md5 45887 45888 45889 45890 45891 ## Implementation details (advanced) 45892 45893 This section explains how various rclone operations work on a hasher remote. 45894 45895 **Disclaimer. This section describes current implementation which can 45896 change in future rclone versions!.** 45897 45898 ### Hashsum command 45899 45900 The \[ga]rclone hashsum\[ga] (or \[ga]md5sum\[ga] or \[ga]sha1sum\[ga]) command will: 45901 45902 1. if requested hash is supported by lower level, just pass it. 45903 2. if object size is below \[ga]auto_size\[ga] then download object and calculate 45904 _requested_ hashes on the fly. 45905 3. if unsupported and the size is big enough, build object \[ga]fingerprint\[ga] 45906 (including size, modtime if supported, first-found _other_ hash if any). 45907 4. if the strict match is found in cache for the requested remote, return 45908 the stored hash. 45909 5. if remote found but fingerprint mismatched, then purge the entry and 45910 proceed to step 6. 45911 6. if remote not found or had no requested hash type or after step 5: 45912 download object, calculate all _supported_ hashes on the fly and store 45913 in cache; return requested hash. 45914 45915 ### Other operations 45916 45917 - whenever a file is uploaded or downloaded **in full**, capture the stream 45918 to calculate all supported hashes on the fly and update database 45919 - server-side \[ga]move\[ga] will update keys of existing cache entries 45920 - \[ga]deletefile\[ga] will remove a single cache entry 45921 - \[ga]purge\[ga] will remove all cache entries under the purged path 45922 45923 Note that setting \[ga]max_age = 0\[ga] will disable checksum caching completely. 45924 45925 If you set \[ga]max_age = off\[ga], checksums in cache will never age, unless you 45926 fully rewrite or delete the file. 45927 45928 ### Cache storage 45929 45930 Cached checksums are stored as \[ga]bolt\[ga] database files under rclone cache 45931 directory, usually \[ga]\[ti]/.cache/rclone/kv/\[ga]. Databases are maintained 45932 one per _base_ backend, named like \[ga]BaseRemote\[ti]hasher.bolt\[ga]. 45933 Checksums for multiple \[ga]alias\[ga]-es into a single base backend 45934 will be stored in the single database. All local paths are treated as 45935 aliases into the \[ga]local\[ga] backend (unless encrypted or chunked) and stored 45936 in \[ga]\[ti]/.cache/rclone/kv/local\[ti]hasher.bolt\[ga]. 45937 Databases can be shared between multiple rclone processes. 45938 45939 # HDFS 45940 45941 [HDFS](https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HdfsDesign.html) is a 45942 distributed file-system, part of the [Apache Hadoop](https://hadoop.apache.org/) framework. 45943 45944 Paths are specified as \[ga]remote:\[ga] or \[ga]remote:path/to/dir\[ga]. 45945 45946 ## Configuration 45947 45948 Here is an example of how to make a remote called \[ga]remote\[ga]. First run: 45949 45950 rclone config 45951 45952 This will guide you through an interactive setup process: 45953 \f[R] 45954 .fi 45955 .PP 45956 No remotes found, make a new one? 45957 n) New remote s) Set configuration password q) Quit config n/s/q> n 45958 name> remote Type of storage to configure. 45959 Enter a string value. 45960 Press Enter for the default (\[dq]\[dq]). 45961 Choose a number from below, or type in your own value [skip] XX / Hadoop 45962 distributed file system \ \[dq]hdfs\[dq] [skip] Storage> hdfs ** See 45963 help for hdfs backend at: https://rclone.org/hdfs/ ** 45964 .PP 45965 hadoop name node and port Enter a string value. 45966 Press Enter for the default (\[dq]\[dq]). 45967 Choose a number from below, or type in your own value 1 / Connect to 45968 host namenode at port 8020 \ \[dq]namenode:8020\[dq] namenode> 45969 namenode.hadoop:8020 hadoop user name Enter a string value. 45970 Press Enter for the default (\[dq]\[dq]). 45971 Choose a number from below, or type in your own value 1 / Connect to 45972 hdfs as root \ \[dq]root\[dq] username> root Edit advanced config? 45973 (y/n) y) Yes n) No (default) y/n> n Remote config -------------------- 45974 [remote] type = hdfs namenode = namenode.hadoop:8020 username = root 45975 -------------------- y) Yes this is OK (default) e) Edit this remote d) 45976 Delete this remote y/e/d> y Current remotes: 45977 .PP 45978 Name Type ==== ==== hadoop hdfs 45979 .IP "e)" 3 45980 Edit existing remote 45981 .IP "f)" 3 45982 New remote 45983 .IP "g)" 3 45984 Delete remote 45985 .IP "h)" 3 45986 Rename remote 45987 .IP "i)" 3 45988 Copy remote 45989 .IP "j)" 3 45990 Set configuration password 45991 .IP "k)" 3 45992 Quit config e/n/d/r/c/s/q> q 45993 .IP 45994 .nf 45995 \f[C] 45996 This remote is called \[ga]remote\[ga] and can now be used like this 45997 45998 See all the top level directories 45999 46000 rclone lsd remote: 46001 46002 List the contents of a directory 46003 46004 rclone ls remote:directory 46005 46006 Sync the remote \[ga]directory\[ga] to \[ga]/home/local/directory\[ga], deleting any excess files. 46007 46008 rclone sync --interactive remote:directory /home/local/directory 46009 46010 ### Setting up your own HDFS instance for testing 46011 46012 You may start with a [manual setup](https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/SingleCluster.html) 46013 or use the docker image from the tests: 46014 46015 If you want to build the docker image 46016 \f[R] 46017 .fi 46018 .PP 46019 git clone https://github.com/artpar/artpar.git cd 46020 rclone/fstest/testserver/images/test-hdfs docker build --rm -t 46021 rclone/test-hdfs . 46022 .IP 46023 .nf 46024 \f[C] 46025 Or you can just use the latest one pushed 46026 \f[R] 46027 .fi 46028 .PP 46029 docker run --rm --name \[dq]rclone-hdfs\[dq] -p 127.0.0.1:9866:9866 -p 46030 127.0.0.1:8020:8020 --hostname \[dq]rclone-hdfs\[dq] rclone/test-hdfs 46031 .IP 46032 .nf 46033 \f[C] 46034 **NB** it need few seconds to startup. 46035 46036 For this docker image the remote needs to be configured like this: 46037 \f[R] 46038 .fi 46039 .PP 46040 [remote] type = hdfs namenode = 127.0.0.1:8020 username = root 46041 .IP 46042 .nf 46043 \f[C] 46044 You can stop this image with \[ga]docker kill rclone-hdfs\[ga] (**NB** it does not use volumes, so all data 46045 uploaded will be lost.) 46046 46047 ### Modification times 46048 46049 Time accurate to 1 second is stored. 46050 46051 ### Checksum 46052 46053 No checksums are implemented. 46054 46055 ### Usage information 46056 46057 You can use the \[ga]rclone about remote:\[ga] command which will display filesystem size and current usage. 46058 46059 ### Restricted filename characters 46060 46061 In addition to the [default restricted characters set](https://rclone.org/overview/#restricted-characters) 46062 the following characters are also replaced: 46063 46064 | Character | Value | Replacement | 46065 | --------- |:-----:|:-----------:| 46066 | : | 0x3A | \[uFF1A] | 46067 46068 Invalid UTF-8 bytes will also be [replaced](https://rclone.org/overview/#invalid-utf8). 46069 46070 46071 ### Standard options 46072 46073 Here are the Standard options specific to hdfs (Hadoop distributed file system). 46074 46075 #### --hdfs-namenode 46076 46077 Hadoop name nodes and ports. 46078 46079 E.g. \[dq]namenode-1:8020,namenode-2:8020,...\[dq] to connect to host namenodes at port 8020. 46080 46081 Properties: 46082 46083 - Config: namenode 46084 - Env Var: RCLONE_HDFS_NAMENODE 46085 - Type: CommaSepList 46086 - Default: 46087 46088 #### --hdfs-username 46089 46090 Hadoop user name. 46091 46092 Properties: 46093 46094 - Config: username 46095 - Env Var: RCLONE_HDFS_USERNAME 46096 - Type: string 46097 - Required: false 46098 - Examples: 46099 - \[dq]root\[dq] 46100 - Connect to hdfs as root. 46101 46102 ### Advanced options 46103 46104 Here are the Advanced options specific to hdfs (Hadoop distributed file system). 46105 46106 #### --hdfs-service-principal-name 46107 46108 Kerberos service principal name for the namenode. 46109 46110 Enables KERBEROS authentication. Specifies the Service Principal Name 46111 (SERVICE/FQDN) for the namenode. E.g. \[rs]\[dq]hdfs/namenode.hadoop.docker\[rs]\[dq] 46112 for namenode running as service \[aq]hdfs\[aq] with FQDN \[aq]namenode.hadoop.docker\[aq]. 46113 46114 Properties: 46115 46116 - Config: service_principal_name 46117 - Env Var: RCLONE_HDFS_SERVICE_PRINCIPAL_NAME 46118 - Type: string 46119 - Required: false 46120 46121 #### --hdfs-data-transfer-protection 46122 46123 Kerberos data transfer protection: authentication|integrity|privacy. 46124 46125 Specifies whether or not authentication, data signature integrity 46126 checks, and wire encryption are required when communicating with 46127 the datanodes. Possible values are \[aq]authentication\[aq], \[aq]integrity\[aq] 46128 and \[aq]privacy\[aq]. Used only with KERBEROS enabled. 46129 46130 Properties: 46131 46132 - Config: data_transfer_protection 46133 - Env Var: RCLONE_HDFS_DATA_TRANSFER_PROTECTION 46134 - Type: string 46135 - Required: false 46136 - Examples: 46137 - \[dq]privacy\[dq] 46138 - Ensure authentication, integrity and encryption enabled. 46139 46140 #### --hdfs-encoding 46141 46142 The encoding for the backend. 46143 46144 See the [encoding section in the overview](https://rclone.org/overview/#encoding) for more info. 46145 46146 Properties: 46147 46148 - Config: encoding 46149 - Env Var: RCLONE_HDFS_ENCODING 46150 - Type: Encoding 46151 - Default: Slash,Colon,Del,Ctl,InvalidUtf8,Dot 46152 46153 #### --hdfs-description 46154 46155 Description of the remote 46156 46157 Properties: 46158 46159 - Config: description 46160 - Env Var: RCLONE_HDFS_DESCRIPTION 46161 - Type: string 46162 - Required: false 46163 46164 46165 46166 ## Limitations 46167 46168 - No server-side \[ga]Move\[ga] or \[ga]DirMove\[ga]. 46169 - Checksums not implemented. 46170 46171 # HiDrive 46172 46173 Paths are specified as \[ga]remote:path\[ga] 46174 46175 Paths may be as deep as required, e.g. \[ga]remote:directory/subdirectory\[ga]. 46176 46177 The initial setup for hidrive involves getting a token from HiDrive 46178 which you need to do in your browser. 46179 \[ga]rclone config\[ga] walks you through it. 46180 46181 ## Configuration 46182 46183 Here is an example of how to make a remote called \[ga]remote\[ga]. First run: 46184 46185 rclone config 46186 46187 This will guide you through an interactive setup process: 46188 \f[R] 46189 .fi 46190 .PP 46191 No remotes found - make a new one n) New remote s) Set configuration 46192 password q) Quit config n/s/q> n name> remote Type of storage to 46193 configure. 46194 Choose a number from below, or type in your own value [snip] XX / 46195 HiDrive \ \[dq]hidrive\[dq] [snip] Storage> hidrive OAuth Client Id - 46196 Leave blank normally. 46197 client_id> OAuth Client Secret - Leave blank normally. 46198 client_secret> Access permissions that rclone should use when requesting 46199 access from HiDrive. 46200 Leave blank normally. 46201 scope_access> Edit advanced config? 46202 y/n> n Use web browser to automatically authenticate rclone with remote? 46203 * Say Y if the machine running rclone has a web browser you can use * 46204 Say N if running rclone on a (remote) machine without web browser access 46205 If not sure try Y. 46206 If Y failed, try N. 46207 y/n> y If your browser doesn\[aq]t open automatically go to the 46208 following link: http://127.0.0.1:53682/auth?state=xxxxxxxxxxxxxxxxxxxxxx 46209 Log in and authorize rclone for access Waiting for code... 46210 Got code -------------------- [remote] type = hidrive token = 46211 {\[dq]access_token\[dq]:\[dq]xxxxxxxxxxxxxxxxxxxx\[dq],\[dq]token_type\[dq]:\[dq]Bearer\[dq],\[dq]refresh_token\[dq]:\[dq]xxxxxxxxxxxxxxxxxxxxxxx\[dq],\[dq]expiry\[dq]:\[dq]xxxxxxxxxxxxxxxxxxxxxxx\[dq]} 46212 -------------------- y) Yes this is OK (default) e) Edit this remote d) 46213 Delete this remote y/e/d> y 46214 .IP 46215 .nf 46216 \f[C] 46217 **You should be aware that OAuth-tokens can be used to access your account 46218 and hence should not be shared with other persons.** 46219 See the [below section](#keeping-your-tokens-safe) for more information. 46220 46221 See the [remote setup docs](https://rclone.org/remote_setup/) for how to set it up on a 46222 machine with no Internet browser available. 46223 46224 Note that rclone runs a webserver on your local machine to collect the 46225 token as returned from HiDrive. This only runs from the moment it opens 46226 your browser to the moment you get back the verification code. 46227 The webserver runs on \[ga]http://127.0.0.1:53682/\[ga]. 46228 If local port \[ga]53682\[ga] is protected by a firewall you may need to temporarily 46229 unblock the firewall to complete authorization. 46230 46231 Once configured you can then use \[ga]rclone\[ga] like this, 46232 46233 List directories in top level of your HiDrive root folder 46234 46235 rclone lsd remote: 46236 46237 List all the files in your HiDrive filesystem 46238 46239 rclone ls remote: 46240 46241 To copy a local directory to a HiDrive directory called backup 46242 46243 rclone copy /home/source remote:backup 46244 46245 ### Keeping your tokens safe 46246 46247 Any OAuth-tokens will be stored by rclone in the remote\[aq]s configuration file as unencrypted text. 46248 Anyone can use a valid refresh-token to access your HiDrive filesystem without knowing your password. 46249 Therefore you should make sure no one else can access your configuration. 46250 46251 It is possible to encrypt rclone\[aq]s configuration file. 46252 You can find information on securing your configuration file by viewing the [configuration encryption docs](https://rclone.org/docs/#configuration-encryption). 46253 46254 ### Invalid refresh token 46255 46256 As can be verified [here](https://developer.hidrive.com/basics-flows/), 46257 each \[ga]refresh_token\[ga] (for Native Applications) is valid for 60 days. 46258 If used to access HiDrivei, its validity will be automatically extended. 46259 46260 This means that if you 46261 46262 * Don\[aq]t use the HiDrive remote for 60 days 46263 46264 then rclone will return an error which includes a text 46265 that implies the refresh token is *invalid* or *expired*. 46266 46267 To fix this you will need to authorize rclone to access your HiDrive account again. 46268 46269 Using 46270 46271 rclone config reconnect remote: 46272 46273 the process is very similar to the process of initial setup exemplified before. 46274 46275 ### Modification times and hashes 46276 46277 HiDrive allows modification times to be set on objects accurate to 1 second. 46278 46279 HiDrive supports [its own hash type](https://static.hidrive.com/dev/0001) 46280 which is used to verify the integrity of file contents after successful transfers. 46281 46282 ### Restricted filename characters 46283 46284 HiDrive cannot store files or folders that include 46285 \[ga]/\[ga] (0x2F) or null-bytes (0x00) in their name. 46286 Any other characters can be used in the names of files or folders. 46287 Additionally, files or folders cannot be named either of the following: \[ga].\[ga] or \[ga]..\[ga] 46288 46289 Therefore rclone will automatically replace these characters, 46290 if files or folders are stored or accessed with such names. 46291 46292 You can read about how this filename encoding works in general 46293 [here](overview/#restricted-filenames). 46294 46295 Keep in mind that HiDrive only supports file or folder names 46296 with a length of 255 characters or less. 46297 46298 ### Transfers 46299 46300 HiDrive limits file sizes per single request to a maximum of 2 GiB. 46301 To allow storage of larger files and allow for better upload performance, 46302 the hidrive backend will use a chunked transfer for files larger than 96 MiB. 46303 Rclone will upload multiple parts/chunks of the file at the same time. 46304 Chunks in the process of being uploaded are buffered in memory, 46305 so you may want to restrict this behaviour on systems with limited resources. 46306 46307 You can customize this behaviour using the following options: 46308 46309 * \[ga]chunk_size\[ga]: size of file parts 46310 * \[ga]upload_cutoff\[ga]: files larger or equal to this in size will use a chunked transfer 46311 * \[ga]upload_concurrency\[ga]: number of file-parts to upload at the same time 46312 46313 See the below section about configuration options for more details. 46314 46315 ### Root folder 46316 46317 You can set the root folder for rclone. 46318 This is the directory that rclone considers to be the root of your HiDrive. 46319 46320 Usually, you will leave this blank, and rclone will use the root of the account. 46321 46322 However, you can set this to restrict rclone to a specific folder hierarchy. 46323 46324 This works by prepending the contents of the \[ga]root_prefix\[ga] option 46325 to any paths accessed by rclone. 46326 For example, the following two ways to access the home directory are equivalent: 46327 46328 rclone lsd --hidrive-root-prefix=\[dq]/users/test/\[dq] remote:path 46329 46330 rclone lsd remote:/users/test/path 46331 46332 See the below section about configuration options for more details. 46333 46334 ### Directory member count 46335 46336 By default, rclone will know the number of directory members contained in a directory. 46337 For example, \[ga]rclone lsd\[ga] uses this information. 46338 46339 The acquisition of this information will result in additional time costs for HiDrive\[aq]s API. 46340 When dealing with large directory structures, it may be desirable to circumvent this time cost, 46341 especially when this information is not explicitly needed. 46342 For this, the \[ga]disable_fetching_member_count\[ga] option can be used. 46343 46344 See the below section about configuration options for more details. 46345 46346 46347 ### Standard options 46348 46349 Here are the Standard options specific to hidrive (HiDrive). 46350 46351 #### --hidrive-client-id 46352 46353 OAuth Client Id. 46354 46355 Leave blank normally. 46356 46357 Properties: 46358 46359 - Config: client_id 46360 - Env Var: RCLONE_HIDRIVE_CLIENT_ID 46361 - Type: string 46362 - Required: false 46363 46364 #### --hidrive-client-secret 46365 46366 OAuth Client Secret. 46367 46368 Leave blank normally. 46369 46370 Properties: 46371 46372 - Config: client_secret 46373 - Env Var: RCLONE_HIDRIVE_CLIENT_SECRET 46374 - Type: string 46375 - Required: false 46376 46377 #### --hidrive-scope-access 46378 46379 Access permissions that rclone should use when requesting access from HiDrive. 46380 46381 Properties: 46382 46383 - Config: scope_access 46384 - Env Var: RCLONE_HIDRIVE_SCOPE_ACCESS 46385 - Type: string 46386 - Default: \[dq]rw\[dq] 46387 - Examples: 46388 - \[dq]rw\[dq] 46389 - Read and write access to resources. 46390 - \[dq]ro\[dq] 46391 - Read-only access to resources. 46392 46393 ### Advanced options 46394 46395 Here are the Advanced options specific to hidrive (HiDrive). 46396 46397 #### --hidrive-token 46398 46399 OAuth Access Token as a JSON blob. 46400 46401 Properties: 46402 46403 - Config: token 46404 - Env Var: RCLONE_HIDRIVE_TOKEN 46405 - Type: string 46406 - Required: false 46407 46408 #### --hidrive-auth-url 46409 46410 Auth server URL. 46411 46412 Leave blank to use the provider defaults. 46413 46414 Properties: 46415 46416 - Config: auth_url 46417 - Env Var: RCLONE_HIDRIVE_AUTH_URL 46418 - Type: string 46419 - Required: false 46420 46421 #### --hidrive-token-url 46422 46423 Token server url. 46424 46425 Leave blank to use the provider defaults. 46426 46427 Properties: 46428 46429 - Config: token_url 46430 - Env Var: RCLONE_HIDRIVE_TOKEN_URL 46431 - Type: string 46432 - Required: false 46433 46434 #### --hidrive-scope-role 46435 46436 User-level that rclone should use when requesting access from HiDrive. 46437 46438 Properties: 46439 46440 - Config: scope_role 46441 - Env Var: RCLONE_HIDRIVE_SCOPE_ROLE 46442 - Type: string 46443 - Default: \[dq]user\[dq] 46444 - Examples: 46445 - \[dq]user\[dq] 46446 - User-level access to management permissions. 46447 - This will be sufficient in most cases. 46448 - \[dq]admin\[dq] 46449 - Extensive access to management permissions. 46450 - \[dq]owner\[dq] 46451 - Full access to management permissions. 46452 46453 #### --hidrive-root-prefix 46454 46455 The root/parent folder for all paths. 46456 46457 Fill in to use the specified folder as the parent for all paths given to the remote. 46458 This way rclone can use any folder as its starting point. 46459 46460 Properties: 46461 46462 - Config: root_prefix 46463 - Env Var: RCLONE_HIDRIVE_ROOT_PREFIX 46464 - Type: string 46465 - Default: \[dq]/\[dq] 46466 - Examples: 46467 - \[dq]/\[dq] 46468 - The topmost directory accessible by rclone. 46469 - This will be equivalent with \[dq]root\[dq] if rclone uses a regular HiDrive user account. 46470 - \[dq]root\[dq] 46471 - The topmost directory of the HiDrive user account 46472 - \[dq]\[dq] 46473 - This specifies that there is no root-prefix for your paths. 46474 - When using this you will always need to specify paths to this remote with a valid parent e.g. \[dq]remote:/path/to/dir\[dq] or \[dq]remote:root/path/to/dir\[dq]. 46475 46476 #### --hidrive-endpoint 46477 46478 Endpoint for the service. 46479 46480 This is the URL that API-calls will be made to. 46481 46482 Properties: 46483 46484 - Config: endpoint 46485 - Env Var: RCLONE_HIDRIVE_ENDPOINT 46486 - Type: string 46487 - Default: \[dq]https://api.hidrive.strato.com/2.1\[dq] 46488 46489 #### --hidrive-disable-fetching-member-count 46490 46491 Do not fetch number of objects in directories unless it is absolutely necessary. 46492 46493 Requests may be faster if the number of objects in subdirectories is not fetched. 46494 46495 Properties: 46496 46497 - Config: disable_fetching_member_count 46498 - Env Var: RCLONE_HIDRIVE_DISABLE_FETCHING_MEMBER_COUNT 46499 - Type: bool 46500 - Default: false 46501 46502 #### --hidrive-chunk-size 46503 46504 Chunksize for chunked uploads. 46505 46506 Any files larger than the configured cutoff (or files of unknown size) will be uploaded in chunks of this size. 46507 46508 The upper limit for this is 2147483647 bytes (about 2.000Gi). 46509 That is the maximum amount of bytes a single upload-operation will support. 46510 Setting this above the upper limit or to a negative value will cause uploads to fail. 46511 46512 Setting this to larger values may increase the upload speed at the cost of using more memory. 46513 It can be set to smaller values smaller to save on memory. 46514 46515 Properties: 46516 46517 - Config: chunk_size 46518 - Env Var: RCLONE_HIDRIVE_CHUNK_SIZE 46519 - Type: SizeSuffix 46520 - Default: 48Mi 46521 46522 #### --hidrive-upload-cutoff 46523 46524 Cutoff/Threshold for chunked uploads. 46525 46526 Any files larger than this will be uploaded in chunks of the configured chunksize. 46527 46528 The upper limit for this is 2147483647 bytes (about 2.000Gi). 46529 That is the maximum amount of bytes a single upload-operation will support. 46530 Setting this above the upper limit will cause uploads to fail. 46531 46532 Properties: 46533 46534 - Config: upload_cutoff 46535 - Env Var: RCLONE_HIDRIVE_UPLOAD_CUTOFF 46536 - Type: SizeSuffix 46537 - Default: 96Mi 46538 46539 #### --hidrive-upload-concurrency 46540 46541 Concurrency for chunked uploads. 46542 46543 This is the upper limit for how many transfers for the same file are running concurrently. 46544 Setting this above to a value smaller than 1 will cause uploads to deadlock. 46545 46546 If you are uploading small numbers of large files over high-speed links 46547 and these uploads do not fully utilize your bandwidth, then increasing 46548 this may help to speed up the transfers. 46549 46550 Properties: 46551 46552 - Config: upload_concurrency 46553 - Env Var: RCLONE_HIDRIVE_UPLOAD_CONCURRENCY 46554 - Type: int 46555 - Default: 4 46556 46557 #### --hidrive-encoding 46558 46559 The encoding for the backend. 46560 46561 See the [encoding section in the overview](https://rclone.org/overview/#encoding) for more info. 46562 46563 Properties: 46564 46565 - Config: encoding 46566 - Env Var: RCLONE_HIDRIVE_ENCODING 46567 - Type: Encoding 46568 - Default: Slash,Dot 46569 46570 #### --hidrive-description 46571 46572 Description of the remote 46573 46574 Properties: 46575 46576 - Config: description 46577 - Env Var: RCLONE_HIDRIVE_DESCRIPTION 46578 - Type: string 46579 - Required: false 46580 46581 46582 46583 ## Limitations 46584 46585 ### Symbolic links 46586 46587 HiDrive is able to store symbolic links (*symlinks*) by design, 46588 for example, when unpacked from a zip archive. 46589 46590 There exists no direct mechanism to manage native symlinks in remotes. 46591 As such this implementation has chosen to ignore any native symlinks present in the remote. 46592 rclone will not be able to access or show any symlinks stored in the hidrive-remote. 46593 This means symlinks cannot be individually removed, copied, or moved, 46594 except when removing, copying, or moving the parent folder. 46595 46596 *This does not affect the \[ga].rclonelink\[ga]-files 46597 that rclone uses to encode and store symbolic links.* 46598 46599 ### Sparse files 46600 46601 It is possible to store sparse files in HiDrive. 46602 46603 Note that copying a sparse file will expand the holes 46604 into null-byte (0x00) regions that will then consume disk space. 46605 Likewise, when downloading a sparse file, 46606 the resulting file will have null-byte regions in the place of file holes. 46607 46608 # HTTP 46609 46610 The HTTP remote is a read only remote for reading files of a 46611 webserver. The webserver should provide file listings which rclone 46612 will read and turn into a remote. This has been tested with common 46613 webservers such as Apache/Nginx/Caddy and will likely work with file 46614 listings from most web servers. (If it doesn\[aq]t then please file an 46615 issue, or send a pull request!) 46616 46617 Paths are specified as \[ga]remote:\[ga] or \[ga]remote:path\[ga]. 46618 46619 The \[ga]remote:\[ga] represents the configured [url](#http-url), and any path following 46620 it will be resolved relative to this url, according to the URL standard. This 46621 means with remote url \[ga]https://beta.rclone.org/branch\[ga] and path \[ga]fix\[ga], the 46622 resolved URL will be \[ga]https://beta.rclone.org/branch/fix\[ga], while with path 46623 \[ga]/fix\[ga] the resolved URL will be \[ga]https://beta.rclone.org/fix\[ga] as the absolute 46624 path is resolved from the root of the domain. 46625 46626 If the path following the \[ga]remote:\[ga] ends with \[ga]/\[ga] it will be assumed to point 46627 to a directory. If the path does not end with \[ga]/\[ga], then a HEAD request is sent 46628 and the response used to decide if it it is treated as a file or a directory 46629 (run with \[ga]-vv\[ga] to see details). When [--http-no-head](#http-no-head) is 46630 specified, a path without ending \[ga]/\[ga] is always assumed to be a file. If rclone 46631 incorrectly assumes the path is a file, the solution is to specify the path with 46632 ending \[ga]/\[ga]. When you know the path is a directory, ending it with \[ga]/\[ga] is always 46633 better as it avoids the initial HEAD request. 46634 46635 To just download a single file it is easier to use 46636 [copyurl](https://rclone.org/commands/rclone_copyurl/). 46637 46638 ## Configuration 46639 46640 Here is an example of how to make a remote called \[ga]remote\[ga]. First 46641 run: 46642 46643 rclone config 46644 46645 This will guide you through an interactive setup process: 46646 \f[R] 46647 .fi 46648 .PP 46649 No remotes found, make a new one? 46650 n) New remote s) Set configuration password q) Quit config n/s/q> n 46651 name> remote Type of storage to configure. 46652 Choose a number from below, or type in your own value [snip] XX / HTTP 46653 \ \[dq]http\[dq] [snip] Storage> http URL of http host to connect to 46654 Choose a number from below, or type in your own value 1 / Connect to 46655 example.com \ \[dq]https://example.com\[dq] url> https://beta.rclone.org 46656 Remote config -------------------- [remote] url = 46657 https://beta.rclone.org -------------------- y) Yes this is OK e) Edit 46658 this remote d) Delete this remote y/e/d> y Current remotes: 46659 .PP 46660 Name Type ==== ==== remote http 46661 .IP "e)" 3 46662 Edit existing remote 46663 .IP "f)" 3 46664 New remote 46665 .IP "g)" 3 46666 Delete remote 46667 .IP "h)" 3 46668 Rename remote 46669 .IP "i)" 3 46670 Copy remote 46671 .IP "j)" 3 46672 Set configuration password 46673 .IP "k)" 3 46674 Quit config e/n/d/r/c/s/q> q 46675 .IP 46676 .nf 46677 \f[C] 46678 This remote is called \[ga]remote\[ga] and can now be used like this 46679 46680 See all the top level directories 46681 46682 rclone lsd remote: 46683 46684 List the contents of a directory 46685 46686 rclone ls remote:directory 46687 46688 Sync the remote \[ga]directory\[ga] to \[ga]/home/local/directory\[ga], deleting any excess files. 46689 46690 rclone sync --interactive remote:directory /home/local/directory 46691 46692 ### Read only 46693 46694 This remote is read only - you can\[aq]t upload files to an HTTP server. 46695 46696 ### Modification times 46697 46698 Most HTTP servers store time accurate to 1 second. 46699 46700 ### Checksum 46701 46702 No checksums are stored. 46703 46704 ### Usage without a config file 46705 46706 Since the http remote only has one config parameter it is easy to use 46707 without a config file: 46708 46709 rclone lsd --http-url https://beta.rclone.org :http: 46710 46711 or: 46712 46713 rclone lsd :http,url=\[aq]https://beta.rclone.org\[aq]: 46714 46715 46716 ### Standard options 46717 46718 Here are the Standard options specific to http (HTTP). 46719 46720 #### --http-url 46721 46722 URL of HTTP host to connect to. 46723 46724 E.g. \[dq]https://example.com\[dq], or \[dq]https://user:pass\[at]example.com\[dq] to use a username and password. 46725 46726 Properties: 46727 46728 - Config: url 46729 - Env Var: RCLONE_HTTP_URL 46730 - Type: string 46731 - Required: true 46732 46733 ### Advanced options 46734 46735 Here are the Advanced options specific to http (HTTP). 46736 46737 #### --http-headers 46738 46739 Set HTTP headers for all transactions. 46740 46741 Use this to set additional HTTP headers for all transactions. 46742 46743 The input format is comma separated list of key,value pairs. Standard 46744 [CSV encoding](https://godoc.org/encoding/csv) may be used. 46745 46746 For example, to set a Cookie use \[aq]Cookie,name=value\[aq], or \[aq]\[dq]Cookie\[dq],\[dq]name=value\[dq]\[aq]. 46747 46748 You can set multiple headers, e.g. \[aq]\[dq]Cookie\[dq],\[dq]name=value\[dq],\[dq]Authorization\[dq],\[dq]xxx\[dq]\[aq]. 46749 46750 Properties: 46751 46752 - Config: headers 46753 - Env Var: RCLONE_HTTP_HEADERS 46754 - Type: CommaSepList 46755 - Default: 46756 46757 #### --http-no-slash 46758 46759 Set this if the site doesn\[aq]t end directories with /. 46760 46761 Use this if your target website does not use / on the end of 46762 directories. 46763 46764 A / on the end of a path is how rclone normally tells the difference 46765 between files and directories. If this flag is set, then rclone will 46766 treat all files with Content-Type: text/html as directories and read 46767 URLs from them rather than downloading them. 46768 46769 Note that this may cause rclone to confuse genuine HTML files with 46770 directories. 46771 46772 Properties: 46773 46774 - Config: no_slash 46775 - Env Var: RCLONE_HTTP_NO_SLASH 46776 - Type: bool 46777 - Default: false 46778 46779 #### --http-no-head 46780 46781 Don\[aq]t use HEAD requests. 46782 46783 HEAD requests are mainly used to find file sizes in dir listing. 46784 If your site is being very slow to load then you can try this option. 46785 Normally rclone does a HEAD request for each potential file in a 46786 directory listing to: 46787 46788 - find its size 46789 - check it really exists 46790 - check to see if it is a directory 46791 46792 If you set this option, rclone will not do the HEAD request. This will mean 46793 that directory listings are much quicker, but rclone won\[aq]t have the times or 46794 sizes of any files, and some files that don\[aq]t exist may be in the listing. 46795 46796 Properties: 46797 46798 - Config: no_head 46799 - Env Var: RCLONE_HTTP_NO_HEAD 46800 - Type: bool 46801 - Default: false 46802 46803 #### --http-description 46804 46805 Description of the remote 46806 46807 Properties: 46808 46809 - Config: description 46810 - Env Var: RCLONE_HTTP_DESCRIPTION 46811 - Type: string 46812 - Required: false 46813 46814 ## Backend commands 46815 46816 Here are the commands specific to the http backend. 46817 46818 Run them with 46819 46820 rclone backend COMMAND remote: 46821 46822 The help below will explain what arguments each command takes. 46823 46824 See the [backend](https://rclone.org/commands/rclone_backend/) command for more 46825 info on how to pass options and arguments. 46826 46827 These can be run on a running backend using the rc command 46828 [backend/command](https://rclone.org/rc/#backend-command). 46829 46830 ### set 46831 46832 Set command for updating the config parameters. 46833 46834 rclone backend set remote: [options] [<arguments>+] 46835 46836 This set command can be used to update the config parameters 46837 for a running http backend. 46838 46839 Usage Examples: 46840 46841 rclone backend set remote: [-o opt_name=opt_value] [-o opt_name2=opt_value2] 46842 rclone rc backend/command command=set fs=remote: [-o opt_name=opt_value] [-o opt_name2=opt_value2] 46843 rclone rc backend/command command=set fs=remote: -o url=https://example.com 46844 46845 The option keys are named as they are in the config file. 46846 46847 This rebuilds the connection to the http backend when it is called with 46848 the new parameters. Only new parameters need be passed as the values 46849 will default to those currently in use. 46850 46851 It doesn\[aq]t return anything. 46852 46853 46854 46855 46856 ## Limitations 46857 46858 \[ga]rclone about\[ga] is not supported by the HTTP backend. Backends without 46859 this capability cannot determine free space for an rclone mount or 46860 use policy \[ga]mfs\[ga] (most free space) as a member of an rclone union 46861 remote. 46862 46863 See [List of backends that do not support rclone about](https://rclone.org/overview/#optional-features) and [rclone about](https://rclone.org/commands/rclone_about/) 46864 46865 # ImageKit 46866 This is a backend for the [ImageKit.io](https://imagekit.io/) storage service. 46867 46868 #### About ImageKit 46869 [ImageKit.io](https://imagekit.io/) provides real-time image and video optimizations, transformations, and CDN delivery. Over 1,000 businesses and 70,000 developers trust ImageKit with their images and videos on the web. 46870 46871 46872 #### Accounts & Pricing 46873 46874 To use this backend, you need to [create an account](https://imagekit.io/registration/) on ImageKit. Start with a free plan with generous usage limits. Then, as your requirements grow, upgrade to a plan that best fits your needs. See [the pricing details](https://imagekit.io/plans). 46875 46876 ## Configuration 46877 46878 Here is an example of making an imagekit configuration. 46879 46880 Firstly create a [ImageKit.io](https://imagekit.io/) account and choose a plan. 46881 46882 You will need to log in and get the \[ga]publicKey\[ga] and \[ga]privateKey\[ga] for your account from the developer section. 46883 46884 Now run 46885 \f[R] 46886 .fi 46887 .PP 46888 rclone config 46889 .IP 46890 .nf 46891 \f[C] 46892 This will guide you through an interactive setup process: 46893 \f[R] 46894 .fi 46895 .PP 46896 No remotes found, make a new one? 46897 n) New remote s) Set configuration password q) Quit config n/s/q> n 46898 .PP 46899 Enter the name for the new remote. 46900 name> imagekit-media-library 46901 .PP 46902 Option Storage. 46903 Type of storage to configure. 46904 Choose a number from below, or type in your own value. 46905 [snip] XX / ImageKit.io \ (imagekit) [snip] Storage> imagekit 46906 .PP 46907 Option endpoint. 46908 You can find your ImageKit.io URL endpoint in your 46909 dashboard (https://imagekit.io/dashboard/developer/api-keys) Enter a 46910 value. 46911 endpoint> https://ik.imagekit.io/imagekit_id 46912 .PP 46913 Option public_key. 46914 You can find your ImageKit.io public key in your 46915 dashboard (https://imagekit.io/dashboard/developer/api-keys) Enter a 46916 value. 46917 public_key> public_**************************** 46918 .PP 46919 Option private_key. 46920 You can find your ImageKit.io private key in your 46921 dashboard (https://imagekit.io/dashboard/developer/api-keys) Enter a 46922 value. 46923 private_key> private_**************************** 46924 .PP 46925 Edit advanced config? 46926 y) Yes n) No (default) y/n> n 46927 .PP 46928 Configuration complete. 46929 Options: - type: imagekit - endpoint: https://ik.imagekit.io/imagekit_id 46930 - public_key: public_**************************** - private_key: 46931 private_**************************** 46932 .PP 46933 Keep this \[dq]imagekit-media-library\[dq] remote? 46934 y) Yes this is OK (default) e) Edit this remote d) Delete this remote 46935 y/e/d> y 46936 .IP 46937 .nf 46938 \f[C] 46939 List directories in the top level of your Media Library 46940 \f[R] 46941 .fi 46942 .PP 46943 rclone lsd imagekit-media-library: 46944 .IP 46945 .nf 46946 \f[C] 46947 Make a new directory. 46948 \f[R] 46949 .fi 46950 .PP 46951 rclone mkdir imagekit-media-library:directory 46952 .IP 46953 .nf 46954 \f[C] 46955 List the contents of a directory. 46956 \f[R] 46957 .fi 46958 .PP 46959 rclone ls imagekit-media-library:directory 46960 .IP 46961 .nf 46962 \f[C] 46963 ### Modified time and hashes 46964 46965 ImageKit does not support modification times or hashes yet. 46966 46967 ### Checksums 46968 46969 No checksums are supported. 46970 46971 46972 ### Standard options 46973 46974 Here are the Standard options specific to imagekit (ImageKit.io). 46975 46976 #### --imagekit-endpoint 46977 46978 You can find your ImageKit.io URL endpoint in your [dashboard](https://imagekit.io/dashboard/developer/api-keys) 46979 46980 Properties: 46981 46982 - Config: endpoint 46983 - Env Var: RCLONE_IMAGEKIT_ENDPOINT 46984 - Type: string 46985 - Required: true 46986 46987 #### --imagekit-public-key 46988 46989 You can find your ImageKit.io public key in your [dashboard](https://imagekit.io/dashboard/developer/api-keys) 46990 46991 Properties: 46992 46993 - Config: public_key 46994 - Env Var: RCLONE_IMAGEKIT_PUBLIC_KEY 46995 - Type: string 46996 - Required: true 46997 46998 #### --imagekit-private-key 46999 47000 You can find your ImageKit.io private key in your [dashboard](https://imagekit.io/dashboard/developer/api-keys) 47001 47002 Properties: 47003 47004 - Config: private_key 47005 - Env Var: RCLONE_IMAGEKIT_PRIVATE_KEY 47006 - Type: string 47007 - Required: true 47008 47009 ### Advanced options 47010 47011 Here are the Advanced options specific to imagekit (ImageKit.io). 47012 47013 #### --imagekit-only-signed 47014 47015 If you have configured \[ga]Restrict unsigned image URLs\[ga] in your dashboard settings, set this to true. 47016 47017 Properties: 47018 47019 - Config: only_signed 47020 - Env Var: RCLONE_IMAGEKIT_ONLY_SIGNED 47021 - Type: bool 47022 - Default: false 47023 47024 #### --imagekit-versions 47025 47026 Include old versions in directory listings. 47027 47028 Properties: 47029 47030 - Config: versions 47031 - Env Var: RCLONE_IMAGEKIT_VERSIONS 47032 - Type: bool 47033 - Default: false 47034 47035 #### --imagekit-upload-tags 47036 47037 Tags to add to the uploaded files, e.g. \[dq]tag1,tag2\[dq]. 47038 47039 Properties: 47040 47041 - Config: upload_tags 47042 - Env Var: RCLONE_IMAGEKIT_UPLOAD_TAGS 47043 - Type: string 47044 - Required: false 47045 47046 #### --imagekit-encoding 47047 47048 The encoding for the backend. 47049 47050 See the [encoding section in the overview](https://rclone.org/overview/#encoding) for more info. 47051 47052 Properties: 47053 47054 - Config: encoding 47055 - Env Var: RCLONE_IMAGEKIT_ENCODING 47056 - Type: Encoding 47057 - Default: Slash,LtGt,DoubleQuote,Dollar,Question,Hash,Percent,BackSlash,Del,Ctl,InvalidUtf8,Dot,SquareBracket 47058 47059 #### --imagekit-description 47060 47061 Description of the remote 47062 47063 Properties: 47064 47065 - Config: description 47066 - Env Var: RCLONE_IMAGEKIT_DESCRIPTION 47067 - Type: string 47068 - Required: false 47069 47070 ### Metadata 47071 47072 Any metadata supported by the underlying remote is read and written. 47073 47074 Here are the possible system metadata items for the imagekit backend. 47075 47076 | Name | Help | Type | Example | Read Only | 47077 |------|------|------|---------|-----------| 47078 | aws-tags | AI generated tags by AWS Rekognition associated with the image | string | tag1,tag2 | **Y** | 47079 | btime | Time of file birth (creation) read from Last-Modified header | RFC 3339 | 2006-01-02T15:04:05.999999999Z07:00 | **Y** | 47080 | custom-coordinates | Custom coordinates of the file | string | 0,0,100,100 | **Y** | 47081 | file-type | Type of the file | string | image | **Y** | 47082 | google-tags | AI generated tags by Google Cloud Vision associated with the image | string | tag1,tag2 | **Y** | 47083 | has-alpha | Whether the image has alpha channel or not | bool | | **Y** | 47084 | height | Height of the image or video in pixels | int | | **Y** | 47085 | is-private-file | Whether the file is private or not | bool | | **Y** | 47086 | size | Size of the object in bytes | int64 | | **Y** | 47087 | tags | Tags associated with the file | string | tag1,tag2 | **Y** | 47088 | width | Width of the image or video in pixels | int | | **Y** | 47089 47090 See the [metadata](https://rclone.org/docs/#metadata) docs for more info. 47091 47092 47093 47094 # Internet Archive 47095 47096 The Internet Archive backend utilizes Items on [archive.org](https://archive.org/) 47097 47098 Refer to [IAS3 API documentation](https://archive.org/services/docs/api/ias3.html) for the API this backend uses. 47099 47100 Paths are specified as \[ga]remote:bucket\[ga] (or \[ga]remote:\[ga] for the \[ga]lsd\[ga] 47101 command.) You may put subdirectories in too, e.g. \[ga]remote:item/path/to/dir\[ga]. 47102 47103 Unlike S3, listing up all items uploaded by you isn\[aq]t supported. 47104 47105 Once you have made a remote, you can use it like this: 47106 47107 Make a new item 47108 47109 rclone mkdir remote:item 47110 47111 List the contents of a item 47112 47113 rclone ls remote:item 47114 47115 Sync \[ga]/home/local/directory\[ga] to the remote item, deleting any excess 47116 files in the item. 47117 47118 rclone sync --interactive /home/local/directory remote:item 47119 47120 ## Notes 47121 Because of Internet Archive\[aq]s architecture, it enqueues write operations (and extra post-processings) in a per-item queue. You can check item\[aq]s queue at https://catalogd.archive.org/history/item-name-here . Because of that, all uploads/deletes will not show up immediately and takes some time to be available. 47122 The per-item queue is enqueued to an another queue, Item Deriver Queue. [You can check the status of Item Deriver Queue here.](https://catalogd.archive.org/catalog.php?whereami=1) This queue has a limit, and it may block you from uploading, or even deleting. You should avoid uploading a lot of small files for better behavior. 47123 47124 You can optionally wait for the server\[aq]s processing to finish, by setting non-zero value to \[ga]wait_archive\[ga] key. 47125 By making it wait, rclone can do normal file comparison. 47126 Make sure to set a large enough value (e.g. \[ga]30m0s\[ga] for smaller files) as it can take a long time depending on server\[aq]s queue. 47127 47128 ## About metadata 47129 This backend supports setting, updating and reading metadata of each file. 47130 The metadata will appear as file metadata on Internet Archive. 47131 However, some fields are reserved by both Internet Archive and rclone. 47132 47133 The following are reserved by Internet Archive: 47134 - \[ga]name\[ga] 47135 - \[ga]source\[ga] 47136 - \[ga]size\[ga] 47137 - \[ga]md5\[ga] 47138 - \[ga]crc32\[ga] 47139 - \[ga]sha1\[ga] 47140 - \[ga]format\[ga] 47141 - \[ga]old_version\[ga] 47142 - \[ga]viruscheck\[ga] 47143 - \[ga]summation\[ga] 47144 47145 Trying to set values to these keys is ignored with a warning. 47146 Only setting \[ga]mtime\[ga] is an exception. Doing so make it the identical behavior as setting ModTime. 47147 47148 rclone reserves all the keys starting with \[ga]rclone-\[ga]. Setting value for these keys will give you warnings, but values are set according to request. 47149 47150 If there are multiple values for a key, only the first one is returned. 47151 This is a limitation of rclone, that supports one value per one key. 47152 It can be triggered when you did a server-side copy. 47153 47154 Reading metadata will also provide custom (non-standard nor reserved) ones. 47155 47156 ## Filtering auto generated files 47157 47158 The Internet Archive automatically creates metadata files after 47159 upload. These can cause problems when doing an \[ga]rclone sync\[ga] as rclone 47160 will try, and fail, to delete them. These metadata files are not 47161 changeable, as they are created by the Internet Archive automatically. 47162 47163 These auto-created files can be excluded from the sync using [metadata 47164 filtering](https://rclone.org/filtering/#metadata). 47165 47166 rclone sync ... --metadata-exclude \[dq]source=metadata\[dq] --metadata-exclude \[dq]format=Metadata\[dq] 47167 47168 Which excludes from the sync any files which have the 47169 \[ga]source=metadata\[ga] or \[ga]format=Metadata\[ga] flags which are added to 47170 Internet Archive auto-created files. 47171 47172 ## Configuration 47173 47174 Here is an example of making an internetarchive configuration. 47175 Most applies to the other providers as well, any differences are described [below](#providers). 47176 47177 First run 47178 47179 rclone config 47180 47181 This will guide you through an interactive setup process. 47182 \f[R] 47183 .fi 47184 .PP 47185 No remotes found, make a new one? 47186 n) New remote s) Set configuration password q) Quit config n/s/q> n 47187 name> remote Option Storage. 47188 Type of storage to configure. 47189 Choose a number from below, or type in your own value. 47190 XX / InternetArchive Items \ (internetarchive) Storage> internetarchive 47191 Option access_key_id. 47192 IAS3 Access Key. 47193 Leave blank for anonymous access. 47194 You can find one here: https://archive.org/account/s3.php Enter a value. 47195 Press Enter to leave empty. 47196 access_key_id> XXXX Option secret_access_key. 47197 IAS3 Secret Key (password). 47198 Leave blank for anonymous access. 47199 Enter a value. 47200 Press Enter to leave empty. 47201 secret_access_key> XXXX Edit advanced config? 47202 y) Yes n) No (default) y/n> y Option endpoint. 47203 IAS3 Endpoint. 47204 Leave blank for default value. 47205 Enter a string value. 47206 Press Enter for the default (https://s3.us.archive.org). 47207 endpoint> Option front_endpoint. 47208 Host of InternetArchive Frontend. 47209 Leave blank for default value. 47210 Enter a string value. 47211 Press Enter for the default (https://archive.org). 47212 front_endpoint> Option disable_checksum. 47213 Don\[aq]t store MD5 checksum with object metadata. 47214 Normally rclone will calculate the MD5 checksum of the input before 47215 uploading it so it can ask the server to check the object against 47216 checksum. 47217 This is great for data integrity checking but can cause long delays for 47218 large files to start uploading. 47219 Enter a boolean value (true or false). 47220 Press Enter for the default (true). 47221 disable_checksum> true Option encoding. 47222 The encoding for the backend. 47223 See the encoding section in the 47224 overview (https://rclone.org/overview/#encoding) for more info. 47225 Enter a encoder.MultiEncoder value. 47226 Press Enter for the default 47227 (Slash,Question,Hash,Percent,Del,Ctl,InvalidUtf8,Dot). 47228 encoding> Edit advanced config? 47229 y) Yes n) No (default) y/n> n -------------------- [remote] type = 47230 internetarchive access_key_id = XXXX secret_access_key = XXXX 47231 -------------------- y) Yes this is OK (default) e) Edit this remote d) 47232 Delete this remote y/e/d> y 47233 .IP 47234 .nf 47235 \f[C] 47236 47237 ### Standard options 47238 47239 Here are the Standard options specific to internetarchive (Internet Archive). 47240 47241 #### --internetarchive-access-key-id 47242 47243 IAS3 Access Key. 47244 47245 Leave blank for anonymous access. 47246 You can find one here: https://archive.org/account/s3.php 47247 47248 Properties: 47249 47250 - Config: access_key_id 47251 - Env Var: RCLONE_INTERNETARCHIVE_ACCESS_KEY_ID 47252 - Type: string 47253 - Required: false 47254 47255 #### --internetarchive-secret-access-key 47256 47257 IAS3 Secret Key (password). 47258 47259 Leave blank for anonymous access. 47260 47261 Properties: 47262 47263 - Config: secret_access_key 47264 - Env Var: RCLONE_INTERNETARCHIVE_SECRET_ACCESS_KEY 47265 - Type: string 47266 - Required: false 47267 47268 ### Advanced options 47269 47270 Here are the Advanced options specific to internetarchive (Internet Archive). 47271 47272 #### --internetarchive-endpoint 47273 47274 IAS3 Endpoint. 47275 47276 Leave blank for default value. 47277 47278 Properties: 47279 47280 - Config: endpoint 47281 - Env Var: RCLONE_INTERNETARCHIVE_ENDPOINT 47282 - Type: string 47283 - Default: \[dq]https://s3.us.archive.org\[dq] 47284 47285 #### --internetarchive-front-endpoint 47286 47287 Host of InternetArchive Frontend. 47288 47289 Leave blank for default value. 47290 47291 Properties: 47292 47293 - Config: front_endpoint 47294 - Env Var: RCLONE_INTERNETARCHIVE_FRONT_ENDPOINT 47295 - Type: string 47296 - Default: \[dq]https://archive.org\[dq] 47297 47298 #### --internetarchive-disable-checksum 47299 47300 Don\[aq]t ask the server to test against MD5 checksum calculated by rclone. 47301 Normally rclone will calculate the MD5 checksum of the input before 47302 uploading it so it can ask the server to check the object against checksum. 47303 This is great for data integrity checking but can cause long delays for 47304 large files to start uploading. 47305 47306 Properties: 47307 47308 - Config: disable_checksum 47309 - Env Var: RCLONE_INTERNETARCHIVE_DISABLE_CHECKSUM 47310 - Type: bool 47311 - Default: true 47312 47313 #### --internetarchive-wait-archive 47314 47315 Timeout for waiting the server\[aq]s processing tasks (specifically archive and book_op) to finish. 47316 Only enable if you need to be guaranteed to be reflected after write operations. 47317 0 to disable waiting. No errors to be thrown in case of timeout. 47318 47319 Properties: 47320 47321 - Config: wait_archive 47322 - Env Var: RCLONE_INTERNETARCHIVE_WAIT_ARCHIVE 47323 - Type: Duration 47324 - Default: 0s 47325 47326 #### --internetarchive-encoding 47327 47328 The encoding for the backend. 47329 47330 See the [encoding section in the overview](https://rclone.org/overview/#encoding) for more info. 47331 47332 Properties: 47333 47334 - Config: encoding 47335 - Env Var: RCLONE_INTERNETARCHIVE_ENCODING 47336 - Type: Encoding 47337 - Default: Slash,LtGt,CrLf,Del,Ctl,InvalidUtf8,Dot 47338 47339 #### --internetarchive-description 47340 47341 Description of the remote 47342 47343 Properties: 47344 47345 - Config: description 47346 - Env Var: RCLONE_INTERNETARCHIVE_DESCRIPTION 47347 - Type: string 47348 - Required: false 47349 47350 ### Metadata 47351 47352 Metadata fields provided by Internet Archive. 47353 If there are multiple values for a key, only the first one is returned. 47354 This is a limitation of Rclone, that supports one value per one key. 47355 47356 Owner is able to add custom keys. Metadata feature grabs all the keys including them. 47357 47358 Here are the possible system metadata items for the internetarchive backend. 47359 47360 | Name | Help | Type | Example | Read Only | 47361 |------|------|------|---------|-----------| 47362 | crc32 | CRC32 calculated by Internet Archive | string | 01234567 | **Y** | 47363 | format | Name of format identified by Internet Archive | string | Comma-Separated Values | **Y** | 47364 | md5 | MD5 hash calculated by Internet Archive | string | 01234567012345670123456701234567 | **Y** | 47365 | mtime | Time of last modification, managed by Rclone | RFC 3339 | 2006-01-02T15:04:05.999999999Z | **Y** | 47366 | name | Full file path, without the bucket part | filename | backend/internetarchive/internetarchive.go | **Y** | 47367 | old_version | Whether the file was replaced and moved by keep-old-version flag | boolean | true | **Y** | 47368 | rclone-ia-mtime | Time of last modification, managed by Internet Archive | RFC 3339 | 2006-01-02T15:04:05.999999999Z | N | 47369 | rclone-mtime | Time of last modification, managed by Rclone | RFC 3339 | 2006-01-02T15:04:05.999999999Z | N | 47370 | rclone-update-track | Random value used by Rclone for tracking changes inside Internet Archive | string | aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa | N | 47371 | sha1 | SHA1 hash calculated by Internet Archive | string | 0123456701234567012345670123456701234567 | **Y** | 47372 | size | File size in bytes | decimal number | 123456 | **Y** | 47373 | source | The source of the file | string | original | **Y** | 47374 | summation | Check https://forum.rclone.org/t/31922 for how it is used | string | md5 | **Y** | 47375 | viruscheck | The last time viruscheck process was run for the file (?) | unixtime | 1654191352 | **Y** | 47376 47377 See the [metadata](https://rclone.org/docs/#metadata) docs for more info. 47378 47379 47380 47381 # Jottacloud 47382 47383 Jottacloud is a cloud storage service provider from a Norwegian company, using its own datacenters 47384 in Norway. In addition to the official service at [jottacloud.com](https://www.jottacloud.com/), 47385 it also provides white-label solutions to different companies, such as: 47386 * Telia 47387 * Telia Cloud (cloud.telia.se) 47388 * Telia Sky (sky.telia.no) 47389 * Tele2 47390 * Tele2 Cloud (mittcloud.tele2.se) 47391 * Onlime 47392 * Onlime Cloud Storage (onlime.dk) 47393 * Elkj\[/o]p (with subsidiaries): 47394 * Elkj\[/o]p Cloud (cloud.elkjop.no) 47395 * Elgiganten Sweden (cloud.elgiganten.se) 47396 * Elgiganten Denmark (cloud.elgiganten.dk) 47397 * Giganti Cloud (cloud.gigantti.fi) 47398 * ELKO Cloud (cloud.elko.is) 47399 47400 Most of the white-label versions are supported by this backend, although may require different 47401 authentication setup - described below. 47402 47403 Paths are specified as \[ga]remote:path\[ga] 47404 47405 Paths may be as deep as required, e.g. \[ga]remote:directory/subdirectory\[ga]. 47406 47407 ## Authentication types 47408 47409 Some of the whitelabel versions uses a different authentication method than the official service, 47410 and you have to choose the correct one when setting up the remote. 47411 47412 ### Standard authentication 47413 47414 The standard authentication method used by the official service (jottacloud.com), as well as 47415 some of the whitelabel services, requires you to generate a single-use personal login token 47416 from the account security settings in the service\[aq]s web interface. Log in to your account, 47417 go to \[dq]Settings\[dq] and then \[dq]Security\[dq], or use the direct link presented to you by rclone when 47418 configuring the remote: <https://www.jottacloud.com/web/secure>. Scroll down to the section 47419 \[dq]Personal login token\[dq], and click the \[dq]Generate\[dq] button. Note that if you are using a 47420 whitelabel service you probably can\[aq]t use the direct link, you need to find the same page in 47421 their dedicated web interface, and also it may be in a different location than described above. 47422 47423 To access your account from multiple instances of rclone, you need to configure each of them 47424 with a separate personal login token. E.g. you create a Jottacloud remote with rclone in one 47425 location, and copy the configuration file to a second location where you also want to run 47426 rclone and access the same remote. Then you need to replace the token for one of them, using 47427 the [config reconnect](https://rclone.org/commands/rclone_config_reconnect/) command, which 47428 requires you to generate a new personal login token and supply as input. If you do not 47429 do this, the token may easily end up being invalidated, resulting in both instances failing 47430 with an error message something along the lines of: 47431 47432 oauth2: cannot fetch token: 400 Bad Request 47433 Response: {\[dq]error\[dq]:\[dq]invalid_grant\[dq],\[dq]error_description\[dq]:\[dq]Stale token\[dq]} 47434 47435 When this happens, you need to replace the token as described above to be able to use your 47436 remote again. 47437 47438 All personal login tokens you have taken into use will be listed in the web interface under 47439 \[dq]My logged in devices\[dq], and from the right side of that list you can click the \[dq]X\[dq] button to 47440 revoke individual tokens. 47441 47442 ### Legacy authentication 47443 47444 If you are using one of the whitelabel versions (e.g. from Elkj\[/o]p) you may not have the option 47445 to generate a CLI token. In this case you\[aq]ll have to use the legacy authentication. To do this select 47446 yes when the setup asks for legacy authentication and enter your username and password. 47447 The rest of the setup is identical to the default setup. 47448 47449 ### Telia Cloud authentication 47450 47451 Similar to other whitelabel versions Telia Cloud doesn\[aq]t offer the option of creating a CLI token, and 47452 additionally uses a separate authentication flow where the username is generated internally. To setup 47453 rclone to use Telia Cloud, choose Telia Cloud authentication in the setup. The rest of the setup is 47454 identical to the default setup. 47455 47456 ### Tele2 Cloud authentication 47457 47458 As Tele2-Com Hem merger was completed this authentication can be used for former Com Hem Cloud and 47459 Tele2 Cloud customers as no support for creating a CLI token exists, and additionally uses a separate 47460 authentication flow where the username is generated internally. To setup rclone to use Tele2 Cloud, 47461 choose Tele2 Cloud authentication in the setup. The rest of the setup is identical to the default setup. 47462 47463 ### Onlime Cloud Storage authentication 47464 47465 Onlime has sold access to Jottacloud proper, while providing localized support to Danish Customers, but 47466 have recently set up their own hosting, transferring their customers from Jottacloud servers to their 47467 own ones. 47468 47469 This, of course, necessitates using their servers for authentication, but otherwise functionality and 47470 architecture seems equivalent to Jottacloud. 47471 47472 To setup rclone to use Onlime Cloud Storage, choose Onlime Cloud authentication in the setup. The rest 47473 of the setup is identical to the default setup. 47474 47475 ## Configuration 47476 47477 Here is an example of how to make a remote called \[ga]remote\[ga] with the default setup. First run: 47478 47479 rclone config 47480 47481 This will guide you through an interactive setup process: 47482 \f[R] 47483 .fi 47484 .PP 47485 No remotes found, make a new one? 47486 n) New remote s) Set configuration password q) Quit config n/s/q> n 47487 name> remote Option Storage. 47488 Type of storage to configure. 47489 Choose a number from below, or type in your own value. 47490 [snip] XX / Jottacloud \ (jottacloud) [snip] Storage> jottacloud Edit 47491 advanced config? 47492 y) Yes n) No (default) y/n> n Option config_type. 47493 Select authentication type. 47494 Choose a number from below, or type in an existing string value. 47495 Press Enter for the default (standard). 47496 / Standard authentication. 47497 1 | Use this if you\[aq]re a normal Jottacloud user. 47498 \ (standard) / Legacy authentication. 47499 2 | This is only required for certain whitelabel versions of Jottacloud 47500 and not recommended for normal users. 47501 \ (legacy) / Telia Cloud authentication. 47502 3 | Use this if you are using Telia Cloud. 47503 \ (telia) / Tele2 Cloud authentication. 47504 4 | Use this if you are using Tele2 Cloud. 47505 \ (tele2) / Onlime Cloud authentication. 47506 5 | Use this if you are using Onlime Cloud. 47507 \ (onlime) config_type> 1 Personal login token. 47508 Generate here: https://www.jottacloud.com/web/secure Login Token> Use a 47509 non-standard device/mountpoint? 47510 Choosing no, the default, will let you access the storage used for the 47511 archive section of the official Jottacloud client. 47512 If you instead want to access the sync or the backup section, for 47513 example, you must choose yes. 47514 y) Yes n) No (default) y/n> y Option config_device. 47515 The device to use. 47516 In standard setup the built-in Jotta device is used, which contains 47517 predefined mountpoints for archive, sync etc. 47518 All other devices are treated as backup devices by the official 47519 Jottacloud client. 47520 You may create a new by entering a unique name. 47521 Choose a number from below, or type in your own string value. 47522 Press Enter for the default (DESKTOP-3H31129). 47523 1 > DESKTOP-3H31129 2 > Jotta config_device> 2 Option config_mountpoint. 47524 The mountpoint to use for the built-in device Jotta. 47525 The standard setup is to use the Archive mountpoint. 47526 Most other mountpoints have very limited support in rclone and should 47527 generally be avoided. 47528 Choose a number from below, or type in an existing string value. 47529 Press Enter for the default (Archive). 47530 1 > Archive 2 > Shared 3 > Sync config_mountpoint> 1 47531 -------------------- [remote] type = jottacloud configVersion = 1 47532 client_id = jottacli client_secret = tokenURL = 47533 https://id.jottacloud.com/auth/realms/jottacloud/protocol/openid-connect/token 47534 token = {........} username = 2940e57271a93d987d6f8a21 device = Jotta 47535 mountpoint = Archive -------------------- y) Yes this is OK (default) e) 47536 Edit this remote d) Delete this remote y/e/d> y 47537 .IP 47538 .nf 47539 \f[C] 47540 Once configured you can then use \[ga]rclone\[ga] like this, 47541 47542 List directories in top level of your Jottacloud 47543 47544 rclone lsd remote: 47545 47546 List all the files in your Jottacloud 47547 47548 rclone ls remote: 47549 47550 To copy a local directory to an Jottacloud directory called backup 47551 47552 rclone copy /home/source remote:backup 47553 47554 ### Devices and Mountpoints 47555 47556 The official Jottacloud client registers a device for each computer you install 47557 it on, and shows them in the backup section of the user interface. For each 47558 folder you select for backup it will create a mountpoint within this device. 47559 A built-in device called Jotta is special, and contains mountpoints Archive, 47560 Sync and some others, used for corresponding features in official clients. 47561 47562 With rclone you\[aq]ll want to use the standard Jotta/Archive device/mountpoint in 47563 most cases. However, you may for example want to access files from the sync or 47564 backup functionality provided by the official clients, and rclone therefore 47565 provides the option to select other devices and mountpoints during config. 47566 47567 You are allowed to create new devices and mountpoints. All devices except the 47568 built-in Jotta device are treated as backup devices by official Jottacloud 47569 clients, and the mountpoints on them are individual backup sets. 47570 47571 With the built-in Jotta device, only existing, built-in, mountpoints can be 47572 selected. In addition to the mentioned Archive and Sync, it may contain 47573 several other mountpoints such as: Latest, Links, Shared and Trash. All of 47574 these are special mountpoints with a different internal representation than 47575 the \[dq]regular\[dq] mountpoints. Rclone will only to a very limited degree support 47576 them. Generally you should avoid these, unless you know what you are doing. 47577 47578 ### --fast-list 47579 47580 This backend supports \[ga]--fast-list\[ga] which allows you to use fewer 47581 transactions in exchange for more memory. See the [rclone 47582 docs](https://rclone.org/docs/#fast-list) for more details. 47583 47584 Note that the implementation in Jottacloud always uses only a single 47585 API request to get the entire list, so for large folders this could 47586 lead to long wait time before the first results are shown. 47587 47588 Note also that with rclone version 1.58 and newer, information about 47589 [MIME types](https://rclone.org/overview/#mime-type) and metadata item [utime](#metadata) 47590 are not available when using \[ga]--fast-list\[ga]. 47591 47592 ### Modification times and hashes 47593 47594 Jottacloud allows modification times to be set on objects accurate to 1 47595 second. These will be used to detect whether objects need syncing or 47596 not. 47597 47598 Jottacloud supports MD5 type hashes, so you can use the \[ga]--checksum\[ga] 47599 flag. 47600 47601 Note that Jottacloud requires the MD5 hash before upload so if the 47602 source does not have an MD5 checksum then the file will be cached 47603 temporarily on disk (in location given by 47604 [--temp-dir](https://rclone.org/docs/#temp-dir-dir)) before it is uploaded. 47605 Small files will be cached in memory - see the 47606 [--jottacloud-md5-memory-limit](#jottacloud-md5-memory-limit) flag. 47607 When uploading from local disk the source checksum is always available, 47608 so this does not apply. Starting with rclone version 1.52 the same is 47609 true for encrypted remotes (in older versions the crypt backend would not 47610 calculate hashes for uploads from local disk, so the Jottacloud 47611 backend had to do it as described above). 47612 47613 ### Restricted filename characters 47614 47615 In addition to the [default restricted characters set](https://rclone.org/overview/#restricted-characters) 47616 the following characters are also replaced: 47617 47618 | Character | Value | Replacement | 47619 | --------- |:-----:|:-----------:| 47620 | \[dq] | 0x22 | \[uFF02] | 47621 | * | 0x2A | \[uFF0A] | 47622 | : | 0x3A | \[uFF1A] | 47623 | < | 0x3C | \[uFF1C] | 47624 | > | 0x3E | \[uFF1E] | 47625 | ? | 0x3F | \[uFF1F] | 47626 | \[rs]| | 0x7C | \[uFF5C] | 47627 47628 Invalid UTF-8 bytes will also be [replaced](https://rclone.org/overview/#invalid-utf8), 47629 as they can\[aq]t be used in XML strings. 47630 47631 ### Deleting files 47632 47633 By default, rclone will send all files to the trash when deleting files. They will be permanently 47634 deleted automatically after 30 days. You may bypass the trash and permanently delete files immediately 47635 by using the [--jottacloud-hard-delete](#jottacloud-hard-delete) flag, or set the equivalent environment variable. 47636 Emptying the trash is supported by the [cleanup](https://rclone.org/commands/rclone_cleanup/) command. 47637 47638 ### Versions 47639 47640 Jottacloud supports file versioning. When rclone uploads a new version of a file it creates a new version of it. 47641 Currently rclone only supports retrieving the current version but older versions can be accessed via the Jottacloud Website. 47642 47643 Versioning can be disabled by \[ga]--jottacloud-no-versions\[ga] option. This is achieved by deleting the remote file prior to uploading 47644 a new version. If the upload the fails no version of the file will be available in the remote. 47645 47646 ### Quota information 47647 47648 To view your current quota you can use the \[ga]rclone about remote:\[ga] 47649 command which will display your usage limit (unless it is unlimited) 47650 and the current usage. 47651 47652 47653 ### Standard options 47654 47655 Here are the Standard options specific to jottacloud (Jottacloud). 47656 47657 #### --jottacloud-client-id 47658 47659 OAuth Client Id. 47660 47661 Leave blank normally. 47662 47663 Properties: 47664 47665 - Config: client_id 47666 - Env Var: RCLONE_JOTTACLOUD_CLIENT_ID 47667 - Type: string 47668 - Required: false 47669 47670 #### --jottacloud-client-secret 47671 47672 OAuth Client Secret. 47673 47674 Leave blank normally. 47675 47676 Properties: 47677 47678 - Config: client_secret 47679 - Env Var: RCLONE_JOTTACLOUD_CLIENT_SECRET 47680 - Type: string 47681 - Required: false 47682 47683 ### Advanced options 47684 47685 Here are the Advanced options specific to jottacloud (Jottacloud). 47686 47687 #### --jottacloud-token 47688 47689 OAuth Access Token as a JSON blob. 47690 47691 Properties: 47692 47693 - Config: token 47694 - Env Var: RCLONE_JOTTACLOUD_TOKEN 47695 - Type: string 47696 - Required: false 47697 47698 #### --jottacloud-auth-url 47699 47700 Auth server URL. 47701 47702 Leave blank to use the provider defaults. 47703 47704 Properties: 47705 47706 - Config: auth_url 47707 - Env Var: RCLONE_JOTTACLOUD_AUTH_URL 47708 - Type: string 47709 - Required: false 47710 47711 #### --jottacloud-token-url 47712 47713 Token server url. 47714 47715 Leave blank to use the provider defaults. 47716 47717 Properties: 47718 47719 - Config: token_url 47720 - Env Var: RCLONE_JOTTACLOUD_TOKEN_URL 47721 - Type: string 47722 - Required: false 47723 47724 #### --jottacloud-md5-memory-limit 47725 47726 Files bigger than this will be cached on disk to calculate the MD5 if required. 47727 47728 Properties: 47729 47730 - Config: md5_memory_limit 47731 - Env Var: RCLONE_JOTTACLOUD_MD5_MEMORY_LIMIT 47732 - Type: SizeSuffix 47733 - Default: 10Mi 47734 47735 #### --jottacloud-trashed-only 47736 47737 Only show files that are in the trash. 47738 47739 This will show trashed files in their original directory structure. 47740 47741 Properties: 47742 47743 - Config: trashed_only 47744 - Env Var: RCLONE_JOTTACLOUD_TRASHED_ONLY 47745 - Type: bool 47746 - Default: false 47747 47748 #### --jottacloud-hard-delete 47749 47750 Delete files permanently rather than putting them into the trash. 47751 47752 Properties: 47753 47754 - Config: hard_delete 47755 - Env Var: RCLONE_JOTTACLOUD_HARD_DELETE 47756 - Type: bool 47757 - Default: false 47758 47759 #### --jottacloud-upload-resume-limit 47760 47761 Files bigger than this can be resumed if the upload fail\[aq]s. 47762 47763 Properties: 47764 47765 - Config: upload_resume_limit 47766 - Env Var: RCLONE_JOTTACLOUD_UPLOAD_RESUME_LIMIT 47767 - Type: SizeSuffix 47768 - Default: 10Mi 47769 47770 #### --jottacloud-no-versions 47771 47772 Avoid server side versioning by deleting files and recreating files instead of overwriting them. 47773 47774 Properties: 47775 47776 - Config: no_versions 47777 - Env Var: RCLONE_JOTTACLOUD_NO_VERSIONS 47778 - Type: bool 47779 - Default: false 47780 47781 #### --jottacloud-encoding 47782 47783 The encoding for the backend. 47784 47785 See the [encoding section in the overview](https://rclone.org/overview/#encoding) for more info. 47786 47787 Properties: 47788 47789 - Config: encoding 47790 - Env Var: RCLONE_JOTTACLOUD_ENCODING 47791 - Type: Encoding 47792 - Default: Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,Del,Ctl,InvalidUtf8,Dot 47793 47794 #### --jottacloud-description 47795 47796 Description of the remote 47797 47798 Properties: 47799 47800 - Config: description 47801 - Env Var: RCLONE_JOTTACLOUD_DESCRIPTION 47802 - Type: string 47803 - Required: false 47804 47805 ### Metadata 47806 47807 Jottacloud has limited support for metadata, currently an extended set of timestamps. 47808 47809 Here are the possible system metadata items for the jottacloud backend. 47810 47811 | Name | Help | Type | Example | Read Only | 47812 |------|------|------|---------|-----------| 47813 | btime | Time of file birth (creation), read from rclone metadata | RFC 3339 | 2006-01-02T15:04:05.999999999Z07:00 | N | 47814 | content-type | MIME type, also known as media type | string | text/plain | **Y** | 47815 | mtime | Time of last modification, read from rclone metadata | RFC 3339 | 2006-01-02T15:04:05.999999999Z07:00 | N | 47816 | utime | Time of last upload, when current revision was created, generated by backend | RFC 3339 | 2006-01-02T15:04:05.999999999Z07:00 | **Y** | 47817 47818 See the [metadata](https://rclone.org/docs/#metadata) docs for more info. 47819 47820 47821 47822 ## Limitations 47823 47824 Note that Jottacloud is case insensitive so you can\[aq]t have a file called 47825 \[dq]Hello.doc\[dq] and one called \[dq]hello.doc\[dq]. 47826 47827 There are quite a few characters that can\[aq]t be in Jottacloud file names. Rclone will map these names to and from an identical 47828 looking unicode equivalent. For example if a file has a ? in it will be mapped to \[uFF1F] instead. 47829 47830 Jottacloud only supports filenames up to 255 characters in length. 47831 47832 ## Troubleshooting 47833 47834 Jottacloud exhibits some inconsistent behaviours regarding deleted files and folders which may cause Copy, Move and DirMove 47835 operations to previously deleted paths to fail. Emptying the trash should help in such cases. 47836 47837 # Koofr 47838 47839 Paths are specified as \[ga]remote:path\[ga] 47840 47841 Paths may be as deep as required, e.g. \[ga]remote:directory/subdirectory\[ga]. 47842 47843 ## Configuration 47844 47845 The initial setup for Koofr involves creating an application password for 47846 rclone. You can do that by opening the Koofr 47847 [web application](https://app.koofr.net/app/admin/preferences/password), 47848 giving the password a nice name like \[ga]rclone\[ga] and clicking on generate. 47849 47850 Here is an example of how to make a remote called \[ga]koofr\[ga]. First run: 47851 47852 rclone config 47853 47854 This will guide you through an interactive setup process: 47855 \f[R] 47856 .fi 47857 .PP 47858 No remotes found, make a new one? 47859 n) New remote s) Set configuration password q) Quit config n/s/q> n 47860 name> koofr Option Storage. 47861 Type of storage to configure. 47862 Choose a number from below, or type in your own value. 47863 [snip] 22 / Koofr, Digi Storage and other Koofr-compatible storage 47864 providers \ (koofr) [snip] Storage> koofr Option provider. 47865 Choose your storage provider. 47866 Choose a number from below, or type in your own value. 47867 Press Enter to leave empty. 47868 1 / Koofr, https://app.koofr.net/ \ (koofr) 2 / Digi Storage, 47869 https://storage.rcs-rds.ro/ \ (digistorage) 3 / Any other Koofr API 47870 compatible storage service \ (other) provider> 1 47871 .PD 0 47872 .P 47873 .PD 47874 Option user. 47875 Your user name. 47876 Enter a value. 47877 user> USERNAME Option password. 47878 Your password for rclone (generate one at 47879 https://app.koofr.net/app/admin/preferences/password). 47880 Choose an alternative below. 47881 y) Yes, type in my own password g) Generate random password y/g> y Enter 47882 the password: password: Confirm the password: password: Edit advanced 47883 config? 47884 y) Yes n) No (default) y/n> n Remote config -------------------- [koofr] 47885 type = koofr provider = koofr user = USERNAME password = *** ENCRYPTED 47886 *** -------------------- y) Yes this is OK (default) e) Edit this remote 47887 d) Delete this remote y/e/d> y 47888 .IP 47889 .nf 47890 \f[C] 47891 You can choose to edit advanced config in order to enter your own service URL 47892 if you use an on-premise or white label Koofr instance, or choose an alternative 47893 mount instead of your primary storage. 47894 47895 Once configured you can then use \[ga]rclone\[ga] like this, 47896 47897 List directories in top level of your Koofr 47898 47899 rclone lsd koofr: 47900 47901 List all the files in your Koofr 47902 47903 rclone ls koofr: 47904 47905 To copy a local directory to an Koofr directory called backup 47906 47907 rclone copy /home/source koofr:backup 47908 47909 ### Restricted filename characters 47910 47911 In addition to the [default restricted characters set](https://rclone.org/overview/#restricted-characters) 47912 the following characters are also replaced: 47913 47914 | Character | Value | Replacement | 47915 | --------- |:-----:|:-----------:| 47916 | \[rs] | 0x5C | \[uFF3C] | 47917 47918 Invalid UTF-8 bytes will also be [replaced](https://rclone.org/overview/#invalid-utf8), 47919 as they can\[aq]t be used in XML strings. 47920 47921 47922 ### Standard options 47923 47924 Here are the Standard options specific to koofr (Koofr, Digi Storage and other Koofr-compatible storage providers). 47925 47926 #### --koofr-provider 47927 47928 Choose your storage provider. 47929 47930 Properties: 47931 47932 - Config: provider 47933 - Env Var: RCLONE_KOOFR_PROVIDER 47934 - Type: string 47935 - Required: false 47936 - Examples: 47937 - \[dq]koofr\[dq] 47938 - Koofr, https://app.koofr.net/ 47939 - \[dq]digistorage\[dq] 47940 - Digi Storage, https://storage.rcs-rds.ro/ 47941 - \[dq]other\[dq] 47942 - Any other Koofr API compatible storage service 47943 47944 #### --koofr-endpoint 47945 47946 The Koofr API endpoint to use. 47947 47948 Properties: 47949 47950 - Config: endpoint 47951 - Env Var: RCLONE_KOOFR_ENDPOINT 47952 - Provider: other 47953 - Type: string 47954 - Required: true 47955 47956 #### --koofr-user 47957 47958 Your user name. 47959 47960 Properties: 47961 47962 - Config: user 47963 - Env Var: RCLONE_KOOFR_USER 47964 - Type: string 47965 - Required: true 47966 47967 #### --koofr-password 47968 47969 Your password for rclone (generate one at https://app.koofr.net/app/admin/preferences/password). 47970 47971 **NB** Input to this must be obscured - see [rclone obscure](https://rclone.org/commands/rclone_obscure/). 47972 47973 Properties: 47974 47975 - Config: password 47976 - Env Var: RCLONE_KOOFR_PASSWORD 47977 - Provider: koofr 47978 - Type: string 47979 - Required: true 47980 47981 ### Advanced options 47982 47983 Here are the Advanced options specific to koofr (Koofr, Digi Storage and other Koofr-compatible storage providers). 47984 47985 #### --koofr-mountid 47986 47987 Mount ID of the mount to use. 47988 47989 If omitted, the primary mount is used. 47990 47991 Properties: 47992 47993 - Config: mountid 47994 - Env Var: RCLONE_KOOFR_MOUNTID 47995 - Type: string 47996 - Required: false 47997 47998 #### --koofr-setmtime 47999 48000 Does the backend support setting modification time. 48001 48002 Set this to false if you use a mount ID that points to a Dropbox or Amazon Drive backend. 48003 48004 Properties: 48005 48006 - Config: setmtime 48007 - Env Var: RCLONE_KOOFR_SETMTIME 48008 - Type: bool 48009 - Default: true 48010 48011 #### --koofr-encoding 48012 48013 The encoding for the backend. 48014 48015 See the [encoding section in the overview](https://rclone.org/overview/#encoding) for more info. 48016 48017 Properties: 48018 48019 - Config: encoding 48020 - Env Var: RCLONE_KOOFR_ENCODING 48021 - Type: Encoding 48022 - Default: Slash,BackSlash,Del,Ctl,InvalidUtf8,Dot 48023 48024 #### --koofr-description 48025 48026 Description of the remote 48027 48028 Properties: 48029 48030 - Config: description 48031 - Env Var: RCLONE_KOOFR_DESCRIPTION 48032 - Type: string 48033 - Required: false 48034 48035 48036 48037 ## Limitations 48038 48039 Note that Koofr is case insensitive so you can\[aq]t have a file called 48040 \[dq]Hello.doc\[dq] and one called \[dq]hello.doc\[dq]. 48041 48042 ## Providers 48043 48044 ### Koofr 48045 48046 This is the original [Koofr](https://koofr.eu) storage provider used as main example and described in the [configuration](#configuration) section above. 48047 48048 ### Digi Storage 48049 48050 [Digi Storage](https://www.digi.ro/servicii/online/digi-storage) is a cloud storage service run by [Digi.ro](https://www.digi.ro/) that 48051 provides a Koofr API. 48052 48053 Here is an example of how to make a remote called \[ga]ds\[ga]. First run: 48054 48055 rclone config 48056 48057 This will guide you through an interactive setup process: 48058 \f[R] 48059 .fi 48060 .PP 48061 No remotes found, make a new one? 48062 n) New remote s) Set configuration password q) Quit config n/s/q> n 48063 name> ds Option Storage. 48064 Type of storage to configure. 48065 Choose a number from below, or type in your own value. 48066 [snip] 22 / Koofr, Digi Storage and other Koofr-compatible storage 48067 providers \ (koofr) [snip] Storage> koofr Option provider. 48068 Choose your storage provider. 48069 Choose a number from below, or type in your own value. 48070 Press Enter to leave empty. 48071 1 / Koofr, https://app.koofr.net/ \ (koofr) 2 / Digi Storage, 48072 https://storage.rcs-rds.ro/ \ (digistorage) 3 / Any other Koofr API 48073 compatible storage service \ (other) provider> 2 Option user. 48074 Your user name. 48075 Enter a value. 48076 user> USERNAME Option password. 48077 Your password for rclone (generate one at 48078 https://storage.rcs-rds.ro/app/admin/preferences/password). 48079 Choose an alternative below. 48080 y) Yes, type in my own password g) Generate random password y/g> y Enter 48081 the password: password: Confirm the password: password: Edit advanced 48082 config? 48083 y) Yes n) No (default) y/n> n -------------------- [ds] type = koofr 48084 provider = digistorage user = USERNAME password = *** ENCRYPTED *** 48085 -------------------- y) Yes this is OK (default) e) Edit this remote d) 48086 Delete this remote y/e/d> y 48087 .IP 48088 .nf 48089 \f[C] 48090 ### Other 48091 48092 You may also want to use another, public or private storage provider that runs a Koofr API compatible service, by simply providing the base URL to connect to. 48093 48094 Here is an example of how to make a remote called \[ga]other\[ga]. First run: 48095 48096 rclone config 48097 48098 This will guide you through an interactive setup process: 48099 \f[R] 48100 .fi 48101 .PP 48102 No remotes found, make a new one? 48103 n) New remote s) Set configuration password q) Quit config n/s/q> n 48104 name> other Option Storage. 48105 Type of storage to configure. 48106 Choose a number from below, or type in your own value. 48107 [snip] 22 / Koofr, Digi Storage and other Koofr-compatible storage 48108 providers \ (koofr) [snip] Storage> koofr Option provider. 48109 Choose your storage provider. 48110 Choose a number from below, or type in your own value. 48111 Press Enter to leave empty. 48112 1 / Koofr, https://app.koofr.net/ \ (koofr) 2 / Digi Storage, 48113 https://storage.rcs-rds.ro/ \ (digistorage) 3 / Any other Koofr API 48114 compatible storage service \ (other) provider> 3 Option endpoint. 48115 The Koofr API endpoint to use. 48116 Enter a value. 48117 endpoint> https://koofr.other.org Option user. 48118 Your user name. 48119 Enter a value. 48120 user> USERNAME Option password. 48121 Your password for rclone (generate one at your service\[aq]s settings 48122 page). 48123 Choose an alternative below. 48124 y) Yes, type in my own password g) Generate random password y/g> y Enter 48125 the password: password: Confirm the password: password: Edit advanced 48126 config? 48127 y) Yes n) No (default) y/n> n -------------------- [other] type = koofr 48128 provider = other endpoint = https://koofr.other.org user = USERNAME 48129 password = *** ENCRYPTED *** -------------------- y) Yes this is OK 48130 (default) e) Edit this remote d) Delete this remote y/e/d> y 48131 .IP 48132 .nf 48133 \f[C] 48134 # Linkbox 48135 48136 Linkbox is [a private cloud drive](https://linkbox.to/). 48137 48138 ## Configuration 48139 48140 Here is an example of making a remote for Linkbox. 48141 48142 First run: 48143 48144 rclone config 48145 48146 This will guide you through an interactive setup process: 48147 \f[R] 48148 .fi 48149 .PP 48150 No remotes found, make a new one? 48151 n) New remote s) Set configuration password q) Quit config n/s/q> n 48152 .PP 48153 Enter name for new remote. 48154 name> remote 48155 .PP 48156 Option Storage. 48157 Type of storage to configure. 48158 Choose a number from below, or type in your own value. 48159 XX / Linkbox \ (linkbox) Storage> XX 48160 .PP 48161 Option token. 48162 Token from https://www.linkbox.to/admin/account Enter a value. 48163 token> testFromCLToken 48164 .PP 48165 Configuration complete. 48166 Options: - type: linkbox - token: XXXXXXXXXXX Keep this 48167 \[dq]linkbox\[dq] remote? 48168 y) Yes this is OK (default) e) Edit this remote d) Delete this remote 48169 y/e/d> y 48170 .IP 48171 .nf 48172 \f[C] 48173 48174 ### Standard options 48175 48176 Here are the Standard options specific to linkbox (Linkbox). 48177 48178 #### --linkbox-token 48179 48180 Token from https://www.linkbox.to/admin/account 48181 48182 Properties: 48183 48184 - Config: token 48185 - Env Var: RCLONE_LINKBOX_TOKEN 48186 - Type: string 48187 - Required: true 48188 48189 ### Advanced options 48190 48191 Here are the Advanced options specific to linkbox (Linkbox). 48192 48193 #### --linkbox-description 48194 48195 Description of the remote 48196 48197 Properties: 48198 48199 - Config: description 48200 - Env Var: RCLONE_LINKBOX_DESCRIPTION 48201 - Type: string 48202 - Required: false 48203 48204 48205 48206 ## Limitations 48207 48208 Invalid UTF-8 bytes will also be [replaced](https://rclone.org/overview/#invalid-utf8), 48209 as they can\[aq]t be used in JSON strings. 48210 48211 # Mail.ru Cloud 48212 48213 [Mail.ru Cloud](https://cloud.mail.ru/) is a cloud storage provided by a Russian internet company [Mail.Ru Group](https://mail.ru). The official desktop client is [Disk-O:](https://disk-o.cloud/en), available on Windows and Mac OS. 48214 48215 ## Features highlights 48216 48217 - Paths may be as deep as required, e.g. \[ga]remote:directory/subdirectory\[ga] 48218 - Files have a \[ga]last modified time\[ga] property, directories don\[aq]t 48219 - Deleted files are by default moved to the trash 48220 - Files and directories can be shared via public links 48221 - Partial uploads or streaming are not supported, file size must be known before upload 48222 - Maximum file size is limited to 2G for a free account, unlimited for paid accounts 48223 - Storage keeps hash for all files and performs transparent deduplication, 48224 the hash algorithm is a modified SHA1 48225 - If a particular file is already present in storage, one can quickly submit file hash 48226 instead of long file upload (this optimization is supported by rclone) 48227 48228 ## Configuration 48229 48230 Here is an example of making a mailru configuration. 48231 48232 First create a Mail.ru Cloud account and choose a tariff. 48233 48234 You will need to log in and create an app password for rclone. Rclone 48235 **will not work** with your normal username and password - it will 48236 give an error like \[ga]oauth2: server response missing access_token\[ga]. 48237 48238 - Click on your user icon in the top right 48239 - Go to Security / \[dq]\[u041F]\[u0430]\[u0440]\[u043E]\[u043B]\[u044C] \[u0438] \[u0431]\[u0435]\[u0437]\[u043E]\[u043F]\[u0430]\[u0441]\[u043D]\[u043E]\[u0441]\[u0442]\[u044C]\[dq] 48240 - Click password for apps / \[dq]\[u041F]\[u0430]\[u0440]\[u043E]\[u043B]\[u0438] \[u0434]\[u043B]\[u044F] \[u0432]\[u043D]\[u0435]\[u0448]\[u043D]\[u0438]\[u0445] \[u043F]\[u0440]\[u0438]\[u043B]\[u043E]\[u0436]\[u0435]\[u043D]\[u0438]\[u0439]\[dq] 48241 - Add the password - give it a name - eg \[dq]rclone\[dq] 48242 - Copy the password and use this password below - your normal login password won\[aq]t work. 48243 48244 Now run 48245 48246 rclone config 48247 48248 This will guide you through an interactive setup process: 48249 \f[R] 48250 .fi 48251 .PP 48252 No remotes found, make a new one? 48253 n) New remote s) Set configuration password q) Quit config n/s/q> n 48254 name> remote Type of storage to configure. 48255 Type of storage to configure. 48256 Enter a string value. 48257 Press Enter for the default (\[dq]\[dq]). 48258 Choose a number from below, or type in your own value [snip] XX / 48259 Mail.ru Cloud \ \[dq]mailru\[dq] [snip] Storage> mailru User name 48260 (usually email) Enter a string value. 48261 Press Enter for the default (\[dq]\[dq]). 48262 user> username\[at]mail.ru Password 48263 .PP 48264 This must be an app password - rclone will not work with your normal 48265 password. 48266 See the Configuration section in the docs for how to make an app 48267 password. 48268 y) Yes type in my own password g) Generate random password y/g> y Enter 48269 the password: password: Confirm the password: password: Skip full upload 48270 if there is another file with same data hash. 48271 This feature is called \[dq]speedup\[dq] or \[dq]put by hash\[dq]. 48272 It is especially efficient in case of generally available files like 48273 popular books, video or audio clips [snip] Enter a boolean value (true 48274 or false). 48275 Press Enter for the default (\[dq]true\[dq]). 48276 Choose a number from below, or type in your own value 1 / Enable 48277 \ \[dq]true\[dq] 2 / Disable \ \[dq]false\[dq] speedup_enable> 1 Edit 48278 advanced config? 48279 (y/n) y) Yes n) No y/n> n Remote config -------------------- [remote] 48280 type = mailru user = username\[at]mail.ru pass = *** ENCRYPTED *** 48281 speedup_enable = true -------------------- y) Yes this is OK e) Edit 48282 this remote d) Delete this remote y/e/d> y 48283 .IP 48284 .nf 48285 \f[C] 48286 Configuration of this backend does not require a local web browser. 48287 You can use the configured backend as shown below: 48288 48289 See top level directories 48290 48291 rclone lsd remote: 48292 48293 Make a new directory 48294 48295 rclone mkdir remote:directory 48296 48297 List the contents of a directory 48298 48299 rclone ls remote:directory 48300 48301 Sync \[ga]/home/local/directory\[ga] to the remote path, deleting any 48302 excess files in the path. 48303 48304 rclone sync --interactive /home/local/directory remote:directory 48305 48306 ### Modification times and hashes 48307 48308 Files support a modification time attribute with up to 1 second precision. 48309 Directories do not have a modification time, which is shown as \[dq]Jan 1 1970\[dq]. 48310 48311 File hashes are supported, with a custom Mail.ru algorithm based on SHA1. 48312 If file size is less than or equal to the SHA1 block size (20 bytes), 48313 its hash is simply its data right-padded with zero bytes. 48314 Hashes of a larger file is computed as a SHA1 of the file data 48315 bytes concatenated with a decimal representation of the data length. 48316 48317 ### Emptying Trash 48318 48319 Removing a file or directory actually moves it to the trash, which is not 48320 visible to rclone but can be seen in a web browser. The trashed file 48321 still occupies part of total quota. If you wish to empty your trash 48322 and free some quota, you can use the \[ga]rclone cleanup remote:\[ga] command, 48323 which will permanently delete all your trashed files. 48324 This command does not take any path arguments. 48325 48326 ### Quota information 48327 48328 To view your current quota you can use the \[ga]rclone about remote:\[ga] 48329 command which will display your usage limit (quota) and the current usage. 48330 48331 ### Restricted filename characters 48332 48333 In addition to the [default restricted characters set](https://rclone.org/overview/#restricted-characters) 48334 the following characters are also replaced: 48335 48336 | Character | Value | Replacement | 48337 | --------- |:-----:|:-----------:| 48338 | \[dq] | 0x22 | \[uFF02] | 48339 | * | 0x2A | \[uFF0A] | 48340 | : | 0x3A | \[uFF1A] | 48341 | < | 0x3C | \[uFF1C] | 48342 | > | 0x3E | \[uFF1E] | 48343 | ? | 0x3F | \[uFF1F] | 48344 | \[rs] | 0x5C | \[uFF3C] | 48345 | \[rs]| | 0x7C | \[uFF5C] | 48346 48347 Invalid UTF-8 bytes will also be [replaced](https://rclone.org/overview/#invalid-utf8), 48348 as they can\[aq]t be used in JSON strings. 48349 48350 48351 ### Standard options 48352 48353 Here are the Standard options specific to mailru (Mail.ru Cloud). 48354 48355 #### --mailru-client-id 48356 48357 OAuth Client Id. 48358 48359 Leave blank normally. 48360 48361 Properties: 48362 48363 - Config: client_id 48364 - Env Var: RCLONE_MAILRU_CLIENT_ID 48365 - Type: string 48366 - Required: false 48367 48368 #### --mailru-client-secret 48369 48370 OAuth Client Secret. 48371 48372 Leave blank normally. 48373 48374 Properties: 48375 48376 - Config: client_secret 48377 - Env Var: RCLONE_MAILRU_CLIENT_SECRET 48378 - Type: string 48379 - Required: false 48380 48381 #### --mailru-user 48382 48383 User name (usually email). 48384 48385 Properties: 48386 48387 - Config: user 48388 - Env Var: RCLONE_MAILRU_USER 48389 - Type: string 48390 - Required: true 48391 48392 #### --mailru-pass 48393 48394 Password. 48395 48396 This must be an app password - rclone will not work with your normal 48397 password. See the Configuration section in the docs for how to make an 48398 app password. 48399 48400 48401 **NB** Input to this must be obscured - see [rclone obscure](https://rclone.org/commands/rclone_obscure/). 48402 48403 Properties: 48404 48405 - Config: pass 48406 - Env Var: RCLONE_MAILRU_PASS 48407 - Type: string 48408 - Required: true 48409 48410 #### --mailru-speedup-enable 48411 48412 Skip full upload if there is another file with same data hash. 48413 48414 This feature is called \[dq]speedup\[dq] or \[dq]put by hash\[dq]. It is especially efficient 48415 in case of generally available files like popular books, video or audio clips, 48416 because files are searched by hash in all accounts of all mailru users. 48417 It is meaningless and ineffective if source file is unique or encrypted. 48418 Please note that rclone may need local memory and disk space to calculate 48419 content hash in advance and decide whether full upload is required. 48420 Also, if rclone does not know file size in advance (e.g. in case of 48421 streaming or partial uploads), it will not even try this optimization. 48422 48423 Properties: 48424 48425 - Config: speedup_enable 48426 - Env Var: RCLONE_MAILRU_SPEEDUP_ENABLE 48427 - Type: bool 48428 - Default: true 48429 - Examples: 48430 - \[dq]true\[dq] 48431 - Enable 48432 - \[dq]false\[dq] 48433 - Disable 48434 48435 ### Advanced options 48436 48437 Here are the Advanced options specific to mailru (Mail.ru Cloud). 48438 48439 #### --mailru-token 48440 48441 OAuth Access Token as a JSON blob. 48442 48443 Properties: 48444 48445 - Config: token 48446 - Env Var: RCLONE_MAILRU_TOKEN 48447 - Type: string 48448 - Required: false 48449 48450 #### --mailru-auth-url 48451 48452 Auth server URL. 48453 48454 Leave blank to use the provider defaults. 48455 48456 Properties: 48457 48458 - Config: auth_url 48459 - Env Var: RCLONE_MAILRU_AUTH_URL 48460 - Type: string 48461 - Required: false 48462 48463 #### --mailru-token-url 48464 48465 Token server url. 48466 48467 Leave blank to use the provider defaults. 48468 48469 Properties: 48470 48471 - Config: token_url 48472 - Env Var: RCLONE_MAILRU_TOKEN_URL 48473 - Type: string 48474 - Required: false 48475 48476 #### --mailru-speedup-file-patterns 48477 48478 Comma separated list of file name patterns eligible for speedup (put by hash). 48479 48480 Patterns are case insensitive and can contain \[aq]*\[aq] or \[aq]?\[aq] meta characters. 48481 48482 Properties: 48483 48484 - Config: speedup_file_patterns 48485 - Env Var: RCLONE_MAILRU_SPEEDUP_FILE_PATTERNS 48486 - Type: string 48487 - Default: \[dq]*.mkv,*.avi,*.mp4,*.mp3,*.zip,*.gz,*.rar,*.pdf\[dq] 48488 - Examples: 48489 - \[dq]\[dq] 48490 - Empty list completely disables speedup (put by hash). 48491 - \[dq]*\[dq] 48492 - All files will be attempted for speedup. 48493 - \[dq]*.mkv,*.avi,*.mp4,*.mp3\[dq] 48494 - Only common audio/video files will be tried for put by hash. 48495 - \[dq]*.zip,*.gz,*.rar,*.pdf\[dq] 48496 - Only common archives or PDF books will be tried for speedup. 48497 48498 #### --mailru-speedup-max-disk 48499 48500 This option allows you to disable speedup (put by hash) for large files. 48501 48502 Reason is that preliminary hashing can exhaust your RAM or disk space. 48503 48504 Properties: 48505 48506 - Config: speedup_max_disk 48507 - Env Var: RCLONE_MAILRU_SPEEDUP_MAX_DISK 48508 - Type: SizeSuffix 48509 - Default: 3Gi 48510 - Examples: 48511 - \[dq]0\[dq] 48512 - Completely disable speedup (put by hash). 48513 - \[dq]1G\[dq] 48514 - Files larger than 1Gb will be uploaded directly. 48515 - \[dq]3G\[dq] 48516 - Choose this option if you have less than 3Gb free on local disk. 48517 48518 #### --mailru-speedup-max-memory 48519 48520 Files larger than the size given below will always be hashed on disk. 48521 48522 Properties: 48523 48524 - Config: speedup_max_memory 48525 - Env Var: RCLONE_MAILRU_SPEEDUP_MAX_MEMORY 48526 - Type: SizeSuffix 48527 - Default: 32Mi 48528 - Examples: 48529 - \[dq]0\[dq] 48530 - Preliminary hashing will always be done in a temporary disk location. 48531 - \[dq]32M\[dq] 48532 - Do not dedicate more than 32Mb RAM for preliminary hashing. 48533 - \[dq]256M\[dq] 48534 - You have at most 256Mb RAM free for hash calculations. 48535 48536 #### --mailru-check-hash 48537 48538 What should copy do if file checksum is mismatched or invalid. 48539 48540 Properties: 48541 48542 - Config: check_hash 48543 - Env Var: RCLONE_MAILRU_CHECK_HASH 48544 - Type: bool 48545 - Default: true 48546 - Examples: 48547 - \[dq]true\[dq] 48548 - Fail with error. 48549 - \[dq]false\[dq] 48550 - Ignore and continue. 48551 48552 #### --mailru-user-agent 48553 48554 HTTP user agent used internally by client. 48555 48556 Defaults to \[dq]rclone/VERSION\[dq] or \[dq]--user-agent\[dq] provided on command line. 48557 48558 Properties: 48559 48560 - Config: user_agent 48561 - Env Var: RCLONE_MAILRU_USER_AGENT 48562 - Type: string 48563 - Required: false 48564 48565 #### --mailru-quirks 48566 48567 Comma separated list of internal maintenance flags. 48568 48569 This option must not be used by an ordinary user. It is intended only to 48570 facilitate remote troubleshooting of backend issues. Strict meaning of 48571 flags is not documented and not guaranteed to persist between releases. 48572 Quirks will be removed when the backend grows stable. 48573 Supported quirks: atomicmkdir binlist unknowndirs 48574 48575 Properties: 48576 48577 - Config: quirks 48578 - Env Var: RCLONE_MAILRU_QUIRKS 48579 - Type: string 48580 - Required: false 48581 48582 #### --mailru-encoding 48583 48584 The encoding for the backend. 48585 48586 See the [encoding section in the overview](https://rclone.org/overview/#encoding) for more info. 48587 48588 Properties: 48589 48590 - Config: encoding 48591 - Env Var: RCLONE_MAILRU_ENCODING 48592 - Type: Encoding 48593 - Default: Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Del,Ctl,InvalidUtf8,Dot 48594 48595 #### --mailru-description 48596 48597 Description of the remote 48598 48599 Properties: 48600 48601 - Config: description 48602 - Env Var: RCLONE_MAILRU_DESCRIPTION 48603 - Type: string 48604 - Required: false 48605 48606 48607 48608 ## Limitations 48609 48610 File size limits depend on your account. A single file size is limited by 2G 48611 for a free account and unlimited for paid tariffs. Please refer to the Mail.ru 48612 site for the total uploaded size limits. 48613 48614 Note that Mailru is case insensitive so you can\[aq]t have a file called 48615 \[dq]Hello.doc\[dq] and one called \[dq]hello.doc\[dq]. 48616 48617 # Mega 48618 48619 [Mega](https://mega.nz/) is a cloud storage and file hosting service 48620 known for its security feature where all files are encrypted locally 48621 before they are uploaded. This prevents anyone (including employees of 48622 Mega) from accessing the files without knowledge of the key used for 48623 encryption. 48624 48625 This is an rclone backend for Mega which supports the file transfer 48626 features of Mega using the same client side encryption. 48627 48628 Paths are specified as \[ga]remote:path\[ga] 48629 48630 Paths may be as deep as required, e.g. \[ga]remote:directory/subdirectory\[ga]. 48631 48632 ## Configuration 48633 48634 Here is an example of how to make a remote called \[ga]remote\[ga]. First run: 48635 48636 rclone config 48637 48638 This will guide you through an interactive setup process: 48639 \f[R] 48640 .fi 48641 .PP 48642 No remotes found, make a new one? 48643 n) New remote s) Set configuration password q) Quit config n/s/q> n 48644 name> remote Type of storage to configure. 48645 Choose a number from below, or type in your own value [snip] XX / Mega 48646 \ \[dq]mega\[dq] [snip] Storage> mega User name user> 48647 you\[at]example.com Password. 48648 y) Yes type in my own password g) Generate random password n) No leave 48649 this optional password blank y/g/n> y Enter the password: password: 48650 Confirm the password: password: Remote config -------------------- 48651 [remote] type = mega user = you\[at]example.com pass = *** ENCRYPTED *** 48652 -------------------- y) Yes this is OK e) Edit this remote d) Delete 48653 this remote y/e/d> y 48654 .IP 48655 .nf 48656 \f[C] 48657 **NOTE:** The encryption keys need to have been already generated after a regular login 48658 via the browser, otherwise attempting to use the credentials in \[ga]rclone\[ga] will fail. 48659 48660 Once configured you can then use \[ga]rclone\[ga] like this, 48661 48662 List directories in top level of your Mega 48663 48664 rclone lsd remote: 48665 48666 List all the files in your Mega 48667 48668 rclone ls remote: 48669 48670 To copy a local directory to an Mega directory called backup 48671 48672 rclone copy /home/source remote:backup 48673 48674 ### Modification times and hashes 48675 48676 Mega does not support modification times or hashes yet. 48677 48678 ### Restricted filename characters 48679 48680 | Character | Value | Replacement | 48681 | --------- |:-----:|:-----------:| 48682 | NUL | 0x00 | \[u2400] | 48683 | / | 0x2F | \[uFF0F] | 48684 48685 Invalid UTF-8 bytes will also be [replaced](https://rclone.org/overview/#invalid-utf8), 48686 as they can\[aq]t be used in JSON strings. 48687 48688 ### Duplicated files 48689 48690 Mega can have two files with exactly the same name and path (unlike a 48691 normal file system). 48692 48693 Duplicated files cause problems with the syncing and you will see 48694 messages in the log about duplicates. 48695 48696 Use \[ga]rclone dedupe\[ga] to fix duplicated files. 48697 48698 ### Failure to log-in 48699 48700 #### Object not found 48701 48702 If you are connecting to your Mega remote for the first time, 48703 to test access and synchronization, you may receive an error such as 48704 \f[R] 48705 .fi 48706 .PP 48707 Failed to create file system for \[dq]my-mega-remote:\[dq]: couldn\[aq]t 48708 login: Object (typically, node or user) not found 48709 .IP 48710 .nf 48711 \f[C] 48712 The diagnostic steps often recommended in the [rclone forum](https://forum.rclone.org/search?q=mega) 48713 start with the **MEGAcmd** utility. Note that this refers to 48714 the official C++ command from https://github.com/meganz/MEGAcmd 48715 and not the go language built command from t3rm1n4l/megacmd 48716 that is no longer maintained. 48717 48718 Follow the instructions for installing MEGAcmd and try accessing 48719 your remote as they recommend. You can establish whether or not 48720 you can log in using MEGAcmd, and obtain diagnostic information 48721 to help you, and search or work with others in the forum. 48722 \f[R] 48723 .fi 48724 .PP 48725 MEGA CMD> login me\[at]example.com Password: Fetching nodes ... 48726 Loading transfers from local cache Login complete as me\[at]example.com 48727 me\[at]example.com:/$ 48728 .IP 48729 .nf 48730 \f[C] 48731 Note that some have found issues with passwords containing special 48732 characters. If you can not log on with rclone, but MEGAcmd logs on 48733 just fine, then consider changing your password temporarily to 48734 pure alphanumeric characters, in case that helps. 48735 48736 48737 #### Repeated commands blocks access 48738 48739 Mega remotes seem to get blocked (reject logins) under \[dq]heavy use\[dq]. 48740 We haven\[aq]t worked out the exact blocking rules but it seems to be 48741 related to fast paced, successive rclone commands. 48742 48743 For example, executing this command 90 times in a row \[ga]rclone link 48744 remote:file\[ga] will cause the remote to become \[dq]blocked\[dq]. This is not an 48745 abnormal situation, for example if you wish to get the public links of 48746 a directory with hundred of files... After more or less a week, the 48747 remote will remote accept rclone logins normally again. 48748 48749 You can mitigate this issue by mounting the remote it with \[ga]rclone 48750 mount\[ga]. This will log-in when mounting and a log-out when unmounting 48751 only. You can also run \[ga]rclone rcd\[ga] and then use \[ga]rclone rc\[ga] to run 48752 the commands over the API to avoid logging in each time. 48753 48754 Rclone does not currently close mega sessions (you can see them in the 48755 web interface), however closing the sessions does not solve the issue. 48756 48757 If you space rclone commands by 3 seconds it will avoid blocking the 48758 remote. We haven\[aq]t identified the exact blocking rules, so perhaps one 48759 could execute the command 80 times without waiting and avoid blocking 48760 by waiting 3 seconds, then continuing... 48761 48762 Note that this has been observed by trial and error and might not be 48763 set in stone. 48764 48765 Other tools seem not to produce this blocking effect, as they use a 48766 different working approach (state-based, using sessionIDs instead of 48767 log-in) which isn\[aq]t compatible with the current stateless rclone 48768 approach. 48769 48770 Note that once blocked, the use of other tools (such as megacmd) is 48771 not a sure workaround: following megacmd login times have been 48772 observed in succession for blocked remote: 7 minutes, 20 min, 30min, 30 48773 min, 30min. Web access looks unaffected though. 48774 48775 Investigation is continuing in relation to workarounds based on 48776 timeouts, pacers, retrials and tpslimits - if you discover something 48777 relevant, please post on the forum. 48778 48779 So, if rclone was working nicely and suddenly you are unable to log-in 48780 and you are sure the user and the password are correct, likely you 48781 have got the remote blocked for a while. 48782 48783 48784 ### Standard options 48785 48786 Here are the Standard options specific to mega (Mega). 48787 48788 #### --mega-user 48789 48790 User name. 48791 48792 Properties: 48793 48794 - Config: user 48795 - Env Var: RCLONE_MEGA_USER 48796 - Type: string 48797 - Required: true 48798 48799 #### --mega-pass 48800 48801 Password. 48802 48803 **NB** Input to this must be obscured - see [rclone obscure](https://rclone.org/commands/rclone_obscure/). 48804 48805 Properties: 48806 48807 - Config: pass 48808 - Env Var: RCLONE_MEGA_PASS 48809 - Type: string 48810 - Required: true 48811 48812 ### Advanced options 48813 48814 Here are the Advanced options specific to mega (Mega). 48815 48816 #### --mega-debug 48817 48818 Output more debug from Mega. 48819 48820 If this flag is set (along with -vv) it will print further debugging 48821 information from the mega backend. 48822 48823 Properties: 48824 48825 - Config: debug 48826 - Env Var: RCLONE_MEGA_DEBUG 48827 - Type: bool 48828 - Default: false 48829 48830 #### --mega-hard-delete 48831 48832 Delete files permanently rather than putting them into the trash. 48833 48834 Normally the mega backend will put all deletions into the trash rather 48835 than permanently deleting them. If you specify this then rclone will 48836 permanently delete objects instead. 48837 48838 Properties: 48839 48840 - Config: hard_delete 48841 - Env Var: RCLONE_MEGA_HARD_DELETE 48842 - Type: bool 48843 - Default: false 48844 48845 #### --mega-use-https 48846 48847 Use HTTPS for transfers. 48848 48849 MEGA uses plain text HTTP connections by default. 48850 Some ISPs throttle HTTP connections, this causes transfers to become very slow. 48851 Enabling this will force MEGA to use HTTPS for all transfers. 48852 HTTPS is normally not necessary since all data is already encrypted anyway. 48853 Enabling it will increase CPU usage and add network overhead. 48854 48855 Properties: 48856 48857 - Config: use_https 48858 - Env Var: RCLONE_MEGA_USE_HTTPS 48859 - Type: bool 48860 - Default: false 48861 48862 #### --mega-encoding 48863 48864 The encoding for the backend. 48865 48866 See the [encoding section in the overview](https://rclone.org/overview/#encoding) for more info. 48867 48868 Properties: 48869 48870 - Config: encoding 48871 - Env Var: RCLONE_MEGA_ENCODING 48872 - Type: Encoding 48873 - Default: Slash,InvalidUtf8,Dot 48874 48875 #### --mega-description 48876 48877 Description of the remote 48878 48879 Properties: 48880 48881 - Config: description 48882 - Env Var: RCLONE_MEGA_DESCRIPTION 48883 - Type: string 48884 - Required: false 48885 48886 48887 48888 ### Process \[ga]killed\[ga] 48889 48890 On accounts with large files or something else, memory usage can significantly increase when executing list/sync instructions. When running on cloud providers (like AWS with EC2), check if the instance type has sufficient memory/CPU to execute the commands. Use the resource monitoring tools to inspect after sending the commands. Look [at this issue](https://forum.rclone.org/t/rclone-with-mega-appears-to-work-only-in-some-accounts/40233/4). 48891 48892 ## Limitations 48893 48894 This backend uses the [go-mega go library](https://github.com/t3rm1n4l/go-mega) which is an opensource 48895 go library implementing the Mega API. There doesn\[aq]t appear to be any 48896 documentation for the mega protocol beyond the [mega C++ SDK](https://github.com/meganz/sdk) source code 48897 so there are likely quite a few errors still remaining in this library. 48898 48899 Mega allows duplicate files which may confuse rclone. 48900 48901 # Memory 48902 48903 The memory backend is an in RAM backend. It does not persist its 48904 data - use the local backend for that. 48905 48906 The memory backend behaves like a bucket-based remote (e.g. like 48907 s3). Because it has no parameters you can just use it with the 48908 \[ga]:memory:\[ga] remote name. 48909 48910 ## Configuration 48911 48912 You can configure it as a remote like this with \[ga]rclone config\[ga] too if 48913 you want to: 48914 \f[R] 48915 .fi 48916 .PP 48917 No remotes found, make a new one? 48918 n) New remote s) Set configuration password q) Quit config n/s/q> n 48919 name> remote Type of storage to configure. 48920 Enter a string value. 48921 Press Enter for the default (\[dq]\[dq]). 48922 Choose a number from below, or type in your own value [snip] XX / Memory 48923 \ \[dq]memory\[dq] [snip] Storage> memory ** See help for memory backend 48924 at: https://rclone.org/memory/ ** 48925 .PP 48926 Remote config 48927 .PP 48928 .TS 48929 tab(@); 48930 l. 48931 T{ 48932 [remote] 48933 T} 48934 T{ 48935 type = memory 48936 T} 48937 .TE 48938 .IP "y)" 3 48939 Yes this is OK (default) 48940 .IP "z)" 3 48941 Edit this remote 48942 .IP "a)" 3 48943 Delete this remote y/e/d> y 48944 .IP 48945 .nf 48946 \f[C] 48947 Because the memory backend isn\[aq]t persistent it is most useful for 48948 testing or with an rclone server or rclone mount, e.g. 48949 48950 rclone mount :memory: /mnt/tmp 48951 rclone serve webdav :memory: 48952 rclone serve sftp :memory: 48953 48954 ### Modification times and hashes 48955 48956 The memory backend supports MD5 hashes and modification times accurate to 1 nS. 48957 48958 ### Restricted filename characters 48959 48960 The memory backend replaces the [default restricted characters 48961 set](https://rclone.org/overview/#restricted-characters). 48962 48963 48964 ### Advanced options 48965 48966 Here are the Advanced options specific to memory (In memory object storage system.). 48967 48968 #### --memory-description 48969 48970 Description of the remote 48971 48972 Properties: 48973 48974 - Config: description 48975 - Env Var: RCLONE_MEMORY_DESCRIPTION 48976 - Type: string 48977 - Required: false 48978 48979 48980 48981 # Akamai NetStorage 48982 48983 Paths are specified as \[ga]remote:\[ga] 48984 You may put subdirectories in too, e.g. \[ga]remote:/path/to/dir\[ga]. 48985 If you have a CP code you can use that as the folder after the domain such as \[rs]<domain>\[rs]/\[rs]<cpcode>\[rs]/\[rs]<internal directories within cpcode>. 48986 48987 For example, this is commonly configured with or without a CP code: 48988 * **With a CP code**. \[ga][your-domain-prefix]-nsu.akamaihd.net/123456/subdirectory/\[ga] 48989 * **Without a CP code**. \[ga][your-domain-prefix]-nsu.akamaihd.net\[ga] 48990 48991 48992 See all buckets 48993 rclone lsd remote: 48994 The initial setup for Netstorage involves getting an account and secret. Use \[ga]rclone config\[ga] to walk you through the setup process. 48995 48996 ## Configuration 48997 48998 Here\[aq]s an example of how to make a remote called \[ga]ns1\[ga]. 48999 49000 1. To begin the interactive configuration process, enter this command: 49001 \f[R] 49002 .fi 49003 .PP 49004 rclone config 49005 .IP 49006 .nf 49007 \f[C] 49008 2. Type \[ga]n\[ga] to create a new remote. 49009 \f[R] 49010 .fi 49011 .IP "n)" 3 49012 New remote 49013 .IP "o)" 3 49014 Delete remote 49015 .IP "p)" 3 49016 Quit config e/n/d/q> n 49017 .IP 49018 .nf 49019 \f[C] 49020 3. For this example, enter \[ga]ns1\[ga] when you reach the name> prompt. 49021 \f[R] 49022 .fi 49023 .PP 49024 name> ns1 49025 .IP 49026 .nf 49027 \f[C] 49028 4. Enter \[ga]netstorage\[ga] as the type of storage to configure. 49029 \f[R] 49030 .fi 49031 .PP 49032 Type of storage to configure. 49033 Enter a string value. 49034 Press Enter for the default (\[dq]\[dq]). 49035 Choose a number from below, or type in your own value XX / NetStorage 49036 \ \[dq]netstorage\[dq] Storage> netstorage 49037 .IP 49038 .nf 49039 \f[C] 49040 5. Select between the HTTP or HTTPS protocol. Most users should choose HTTPS, which is the default. HTTP is provided primarily for debugging purposes. 49041 49042 \f[R] 49043 .fi 49044 .PP 49045 Enter a string value. 49046 Press Enter for the default (\[dq]\[dq]). 49047 Choose a number from below, or type in your own value 1 / HTTP protocol 49048 \ \[dq]http\[dq] 2 / HTTPS protocol \ \[dq]https\[dq] protocol> 1 49049 .IP 49050 .nf 49051 \f[C] 49052 6. Specify your NetStorage host, CP code, and any necessary content paths using this format: \[ga]<domain>/<cpcode>/<content>/\[ga] 49053 \f[R] 49054 .fi 49055 .PP 49056 Enter a string value. 49057 Press Enter for the default (\[dq]\[dq]). 49058 host> baseball-nsu.akamaihd.net/123456/content/ 49059 .IP 49060 .nf 49061 \f[C] 49062 7. Set the netstorage account name 49063 \f[R] 49064 .fi 49065 .PP 49066 Enter a string value. 49067 Press Enter for the default (\[dq]\[dq]). 49068 account> username 49069 .IP 49070 .nf 49071 \f[C] 49072 8. Set the Netstorage account secret/G2O key which will be used for authentication purposes. Select the \[ga]y\[ga] option to set your own password then enter your secret. 49073 Note: The secret is stored in the \[ga]rclone.conf\[ga] file with hex-encoded encryption. 49074 \f[R] 49075 .fi 49076 .IP "y)" 3 49077 Yes type in my own password 49078 .IP "z)" 3 49079 Generate random password y/g> y Enter the password: password: Confirm 49080 the password: password: 49081 .IP 49082 .nf 49083 \f[C] 49084 9. View the summary and confirm your remote configuration. 49085 \f[R] 49086 .fi 49087 .PP 49088 [ns1] type = netstorage protocol = http host = 49089 baseball-nsu.akamaihd.net/123456/content/ account = username secret = 49090 *** ENCRYPTED *** -------------------- y) Yes this is OK (default) e) 49091 Edit this remote d) Delete this remote y/e/d> y 49092 .IP 49093 .nf 49094 \f[C] 49095 This remote is called \[ga]ns1\[ga] and can now be used. 49096 49097 ## Example operations 49098 49099 Get started with rclone and NetStorage with these examples. For additional rclone commands, visit https://rclone.org/commands/. 49100 49101 ### See contents of a directory in your project 49102 49103 rclone lsd ns1:/974012/testing/ 49104 49105 ### Sync the contents local with remote 49106 49107 rclone sync . ns1:/974012/testing/ 49108 49109 ### Upload local content to remote 49110 rclone copy notes.txt ns1:/974012/testing/ 49111 49112 ### Delete content on remote 49113 rclone delete ns1:/974012/testing/notes.txt 49114 49115 ### Move or copy content between CP codes. 49116 49117 Your credentials must have access to two CP codes on the same remote. You can\[aq]t perform operations between different remotes. 49118 49119 rclone move ns1:/974012/testing/notes.txt ns1:/974450/testing2/ 49120 49121 ## Features 49122 49123 ### Symlink Support 49124 49125 The Netstorage backend changes the rclone \[ga]--links, -l\[ga] behavior. When uploading, instead of creating the .rclonelink file, use the \[dq]symlink\[dq] API in order to create the corresponding symlink on the remote. The .rclonelink file will not be created, the upload will be intercepted and only the symlink file that matches the source file name with no suffix will be created on the remote. 49126 49127 This will effectively allow commands like copy/copyto, move/moveto and sync to upload from local to remote and download from remote to local directories with symlinks. Due to internal rclone limitations, it is not possible to upload an individual symlink file to any remote backend. You can always use the \[dq]backend symlink\[dq] command to create a symlink on the NetStorage server, refer to \[dq]symlink\[dq] section below. 49128 49129 Individual symlink files on the remote can be used with the commands like \[dq]cat\[dq] to print the destination name, or \[dq]delete\[dq] to delete symlink, or copy, copy/to and move/moveto to download from the remote to local. Note: individual symlink files on the remote should be specified including the suffix .rclonelink. 49130 49131 **Note**: No file with the suffix .rclonelink should ever exist on the server since it is not possible to actually upload/create a file with .rclonelink suffix with rclone, it can only exist if it is manually created through a non-rclone method on the remote. 49132 49133 ### Implicit vs. Explicit Directories 49134 49135 With NetStorage, directories can exist in one of two forms: 49136 49137 1. **Explicit Directory**. This is an actual, physical directory that you have created in a storage group. 49138 2. **Implicit Directory**. This refers to a directory within a path that has not been physically created. For example, during upload of a file, nonexistent subdirectories can be specified in the target path. NetStorage creates these as \[dq]implicit.\[dq] While the directories aren\[aq]t physically created, they exist implicitly and the noted path is connected with the uploaded file. 49139 49140 Rclone will intercept all file uploads and mkdir commands for the NetStorage remote and will explicitly issue the mkdir command for each directory in the uploading path. This will help with the interoperability with the other Akamai services such as SFTP and the Content Management Shell (CMShell). Rclone will not guarantee correctness of operations with implicit directories which might have been created as a result of using an upload API directly. 49141 49142 ### \[ga]--fast-list\[ga] / ListR support 49143 49144 NetStorage remote supports the ListR feature by using the \[dq]list\[dq] NetStorage API action to return a lexicographical list of all objects within the specified CP code, recursing into subdirectories as they\[aq]re encountered. 49145 49146 * **Rclone will use the ListR method for some commands by default**. Commands such as \[ga]lsf -R\[ga] will use ListR by default. To disable this, include the \[ga]--disable listR\[ga] option to use the non-recursive method of listing objects. 49147 49148 * **Rclone will not use the ListR method for some commands**. Commands such as \[ga]sync\[ga] don\[aq]t use ListR by default. To force using the ListR method, include the \[ga]--fast-list\[ga] option. 49149 49150 There are pros and cons of using the ListR method, refer to [rclone documentation](https://rclone.org/docs/#fast-list). In general, the sync command over an existing deep tree on the remote will run faster with the \[dq]--fast-list\[dq] flag but with extra memory usage as a side effect. It might also result in higher CPU utilization but the whole task can be completed faster. 49151 49152 **Note**: There is a known limitation that \[dq]lsf -R\[dq] will display number of files in the directory and directory size as -1 when ListR method is used. The workaround is to pass \[dq]--disable listR\[dq] flag if these numbers are important in the output. 49153 49154 ### Purge 49155 49156 NetStorage remote supports the purge feature by using the \[dq]quick-delete\[dq] NetStorage API action. The quick-delete action is disabled by default for security reasons and can be enabled for the account through the Akamai portal. Rclone will first try to use quick-delete action for the purge command and if this functionality is disabled then will fall back to a standard delete method. 49157 49158 **Note**: Read the [NetStorage Usage API](https://learn.akamai.com/en-us/webhelp/netstorage/netstorage-http-api-developer-guide/GUID-15836617-9F50-405A-833C-EA2556756A30.html) for considerations when using \[dq]quick-delete\[dq]. In general, using quick-delete method will not delete the tree immediately and objects targeted for quick-delete may still be accessible. 49159 49160 49161 ### Standard options 49162 49163 Here are the Standard options specific to netstorage (Akamai NetStorage). 49164 49165 #### --netstorage-host 49166 49167 Domain+path of NetStorage host to connect to. 49168 49169 Format should be \[ga]<domain>/<internal folders>\[ga] 49170 49171 Properties: 49172 49173 - Config: host 49174 - Env Var: RCLONE_NETSTORAGE_HOST 49175 - Type: string 49176 - Required: true 49177 49178 #### --netstorage-account 49179 49180 Set the NetStorage account name 49181 49182 Properties: 49183 49184 - Config: account 49185 - Env Var: RCLONE_NETSTORAGE_ACCOUNT 49186 - Type: string 49187 - Required: true 49188 49189 #### --netstorage-secret 49190 49191 Set the NetStorage account secret/G2O key for authentication. 49192 49193 Please choose the \[aq]y\[aq] option to set your own password then enter your secret. 49194 49195 **NB** Input to this must be obscured - see [rclone obscure](https://rclone.org/commands/rclone_obscure/). 49196 49197 Properties: 49198 49199 - Config: secret 49200 - Env Var: RCLONE_NETSTORAGE_SECRET 49201 - Type: string 49202 - Required: true 49203 49204 ### Advanced options 49205 49206 Here are the Advanced options specific to netstorage (Akamai NetStorage). 49207 49208 #### --netstorage-protocol 49209 49210 Select between HTTP or HTTPS protocol. 49211 49212 Most users should choose HTTPS, which is the default. 49213 HTTP is provided primarily for debugging purposes. 49214 49215 Properties: 49216 49217 - Config: protocol 49218 - Env Var: RCLONE_NETSTORAGE_PROTOCOL 49219 - Type: string 49220 - Default: \[dq]https\[dq] 49221 - Examples: 49222 - \[dq]http\[dq] 49223 - HTTP protocol 49224 - \[dq]https\[dq] 49225 - HTTPS protocol 49226 49227 #### --netstorage-description 49228 49229 Description of the remote 49230 49231 Properties: 49232 49233 - Config: description 49234 - Env Var: RCLONE_NETSTORAGE_DESCRIPTION 49235 - Type: string 49236 - Required: false 49237 49238 ## Backend commands 49239 49240 Here are the commands specific to the netstorage backend. 49241 49242 Run them with 49243 49244 rclone backend COMMAND remote: 49245 49246 The help below will explain what arguments each command takes. 49247 49248 See the [backend](https://rclone.org/commands/rclone_backend/) command for more 49249 info on how to pass options and arguments. 49250 49251 These can be run on a running backend using the rc command 49252 [backend/command](https://rclone.org/rc/#backend-command). 49253 49254 ### du 49255 49256 Return disk usage information for a specified directory 49257 49258 rclone backend du remote: [options] [<arguments>+] 49259 49260 The usage information returned, includes the targeted directory as well as all 49261 files stored in any sub-directories that may exist. 49262 49263 ### symlink 49264 49265 You can create a symbolic link in ObjectStore with the symlink action. 49266 49267 rclone backend symlink remote: [options] [<arguments>+] 49268 49269 The desired path location (including applicable sub-directories) ending in 49270 the object that will be the target of the symlink (for example, /links/mylink). 49271 Include the file extension for the object, if applicable. 49272 \[ga]rclone backend symlink <src> <path>\[ga] 49273 49274 49275 49276 # Microsoft Azure Blob Storage 49277 49278 Paths are specified as \[ga]remote:container\[ga] (or \[ga]remote:\[ga] for the \[ga]lsd\[ga] 49279 command.) You may put subdirectories in too, e.g. 49280 \[ga]remote:container/path/to/dir\[ga]. 49281 49282 ## Configuration 49283 49284 Here is an example of making a Microsoft Azure Blob Storage 49285 configuration. For a remote called \[ga]remote\[ga]. First run: 49286 49287 rclone config 49288 49289 This will guide you through an interactive setup process: 49290 \f[R] 49291 .fi 49292 .PP 49293 No remotes found, make a new one? 49294 n) New remote s) Set configuration password q) Quit config n/s/q> n 49295 name> remote Type of storage to configure. 49296 Choose a number from below, or type in your own value [snip] XX / 49297 Microsoft Azure Blob Storage \ \[dq]azureblob\[dq] [snip] Storage> 49298 azureblob Storage Account Name account> account_name Storage Account Key 49299 key> base64encodedkey== Endpoint for the service - leave blank normally. 49300 endpoint> Remote config -------------------- [remote] account = 49301 account_name key = base64encodedkey== endpoint = -------------------- y) 49302 Yes this is OK e) Edit this remote d) Delete this remote y/e/d> y 49303 .IP 49304 .nf 49305 \f[C] 49306 See all containers 49307 49308 rclone lsd remote: 49309 49310 Make a new container 49311 49312 rclone mkdir remote:container 49313 49314 List the contents of a container 49315 49316 rclone ls remote:container 49317 49318 Sync \[ga]/home/local/directory\[ga] to the remote container, deleting any excess 49319 files in the container. 49320 49321 rclone sync --interactive /home/local/directory remote:container 49322 49323 ### --fast-list 49324 49325 This remote supports \[ga]--fast-list\[ga] which allows you to use fewer 49326 transactions in exchange for more memory. See the [rclone 49327 docs](https://rclone.org/docs/#fast-list) for more details. 49328 49329 ### Modification times and hashes 49330 49331 The modification time is stored as metadata on the object with the 49332 \[ga]mtime\[ga] key. It is stored using RFC3339 Format time with nanosecond 49333 precision. The metadata is supplied during directory listings so 49334 there is no performance overhead to using it. 49335 49336 If you wish to use the Azure standard \[ga]LastModified\[ga] time stored on 49337 the object as the modified time, then use the \[ga]--use-server-modtime\[ga] 49338 flag. Note that rclone can\[aq]t set \[ga]LastModified\[ga], so using the 49339 \[ga]--update\[ga] flag when syncing is recommended if using 49340 \[ga]--use-server-modtime\[ga]. 49341 49342 MD5 hashes are stored with blobs. However blobs that were uploaded in 49343 chunks only have an MD5 if the source remote was capable of MD5 49344 hashes, e.g. the local disk. 49345 49346 ### Performance 49347 49348 When uploading large files, increasing the value of 49349 \[ga]--azureblob-upload-concurrency\[ga] will increase performance at the cost 49350 of using more memory. The default of 16 is set quite conservatively to 49351 use less memory. It maybe be necessary raise it to 64 or higher to 49352 fully utilize a 1 GBit/s link with a single file transfer. 49353 49354 ### Restricted filename characters 49355 49356 In addition to the [default restricted characters set](https://rclone.org/overview/#restricted-characters) 49357 the following characters are also replaced: 49358 49359 | Character | Value | Replacement | 49360 | --------- |:-----:|:-----------:| 49361 | / | 0x2F | \[uFF0F] | 49362 | \[rs] | 0x5C | \[uFF3C] | 49363 49364 File names can also not end with the following characters. 49365 These only get replaced if they are the last character in the name: 49366 49367 | Character | Value | Replacement | 49368 | --------- |:-----:|:-----------:| 49369 | . | 0x2E | \[uFF0E] | 49370 49371 Invalid UTF-8 bytes will also be [replaced](https://rclone.org/overview/#invalid-utf8), 49372 as they can\[aq]t be used in JSON strings. 49373 49374 ### Authentication {#authentication} 49375 49376 There are a number of ways of supplying credentials for Azure Blob 49377 Storage. Rclone tries them in the order of the sections below. 49378 49379 #### Env Auth 49380 49381 If the \[ga]env_auth\[ga] config parameter is \[ga]true\[ga] then rclone will pull 49382 credentials from the environment or runtime. 49383 49384 It tries these authentication methods in this order: 49385 49386 1. Environment Variables 49387 2. Managed Service Identity Credentials 49388 3. Azure CLI credentials (as used by the az tool) 49389 49390 These are described in the following sections 49391 49392 ##### Env Auth: 1. Environment Variables 49393 49394 If \[ga]env_auth\[ga] is set and environment variables are present rclone 49395 authenticates a service principal with a secret or certificate, or a 49396 user with a password, depending on which environment variable are set. 49397 It reads configuration from these variables, in the following order: 49398 49399 1. Service principal with client secret 49400 - \[ga]AZURE_TENANT_ID\[ga]: ID of the service principal\[aq]s tenant. Also called its \[dq]directory\[dq] ID. 49401 - \[ga]AZURE_CLIENT_ID\[ga]: the service principal\[aq]s client ID 49402 - \[ga]AZURE_CLIENT_SECRET\[ga]: one of the service principal\[aq]s client secrets 49403 2. Service principal with certificate 49404 - \[ga]AZURE_TENANT_ID\[ga]: ID of the service principal\[aq]s tenant. Also called its \[dq]directory\[dq] ID. 49405 - \[ga]AZURE_CLIENT_ID\[ga]: the service principal\[aq]s client ID 49406 - \[ga]AZURE_CLIENT_CERTIFICATE_PATH\[ga]: path to a PEM or PKCS12 certificate file including the private key. 49407 - \[ga]AZURE_CLIENT_CERTIFICATE_PASSWORD\[ga]: (optional) password for the certificate file. 49408 - \[ga]AZURE_CLIENT_SEND_CERTIFICATE_CHAIN\[ga]: (optional) Specifies whether an authentication request will include an x5c header to support subject name / issuer based authentication. When set to \[dq]true\[dq] or \[dq]1\[dq], authentication requests include the x5c header. 49409 3. User with username and password 49410 - \[ga]AZURE_TENANT_ID\[ga]: (optional) tenant to authenticate in. Defaults to \[dq]organizations\[dq]. 49411 - \[ga]AZURE_CLIENT_ID\[ga]: client ID of the application the user will authenticate to 49412 - \[ga]AZURE_USERNAME\[ga]: a username (usually an email address) 49413 - \[ga]AZURE_PASSWORD\[ga]: the user\[aq]s password 49414 4. Workload Identity 49415 - \[ga]AZURE_TENANT_ID\[ga]: Tenant to authenticate in. 49416 - \[ga]AZURE_CLIENT_ID\[ga]: Client ID of the application the user will authenticate to. 49417 - \[ga]AZURE_FEDERATED_TOKEN_FILE\[ga]: Path to projected service account token file. 49418 - \[ga]AZURE_AUTHORITY_HOST\[ga]: Authority of an Azure Active Directory endpoint (default: login.microsoftonline.com). 49419 49420 49421 ##### Env Auth: 2. Managed Service Identity Credentials 49422 49423 When using Managed Service Identity if the VM(SS) on which this 49424 program is running has a system-assigned identity, it will be used by 49425 default. If the resource has no system-assigned but exactly one 49426 user-assigned identity, the user-assigned identity will be used by 49427 default. 49428 49429 If the resource has multiple user-assigned identities you will need to 49430 unset \[ga]env_auth\[ga] and set \[ga]use_msi\[ga] instead. See the [\[ga]use_msi\[ga] 49431 section](#use_msi). 49432 49433 ##### Env Auth: 3. Azure CLI credentials (as used by the az tool) 49434 49435 Credentials created with the \[ga]az\[ga] tool can be picked up using \[ga]env_auth\[ga]. 49436 49437 For example if you were to login with a service principal like this: 49438 49439 az login --service-principal -u XXX -p XXX --tenant XXX 49440 49441 Then you could access rclone resources like this: 49442 49443 rclone lsf :azureblob,env_auth,account=ACCOUNT:CONTAINER 49444 49445 Or 49446 49447 rclone lsf --azureblob-env-auth --azureblob-account=ACCOUNT :azureblob:CONTAINER 49448 49449 Which is analogous to using the \[ga]az\[ga] tool: 49450 49451 az storage blob list --container-name CONTAINER --account-name ACCOUNT --auth-mode login 49452 49453 #### Account and Shared Key 49454 49455 This is the most straight forward and least flexible way. Just fill 49456 in the \[ga]account\[ga] and \[ga]key\[ga] lines and leave the rest blank. 49457 49458 #### SAS URL 49459 49460 This can be an account level SAS URL or container level SAS URL. 49461 49462 To use it leave \[ga]account\[ga] and \[ga]key\[ga] blank and fill in \[ga]sas_url\[ga]. 49463 49464 An account level SAS URL or container level SAS URL can be obtained 49465 from the Azure portal or the Azure Storage Explorer. To get a 49466 container level SAS URL right click on a container in the Azure Blob 49467 explorer in the Azure portal. 49468 49469 If you use a container level SAS URL, rclone operations are permitted 49470 only on a particular container, e.g. 49471 49472 rclone ls azureblob:container 49473 49474 You can also list the single container from the root. This will only 49475 show the container specified by the SAS URL. 49476 49477 $ rclone lsd azureblob: 49478 container/ 49479 49480 Note that you can\[aq]t see or access any other containers - this will 49481 fail 49482 49483 rclone ls azureblob:othercontainer 49484 49485 Container level SAS URLs are useful for temporarily allowing third 49486 parties access to a single container or putting credentials into an 49487 untrusted environment such as a CI build server. 49488 49489 #### Service principal with client secret 49490 49491 If these variables are set, rclone will authenticate with a service principal with a client secret. 49492 49493 - \[ga]tenant\[ga]: ID of the service principal\[aq]s tenant. Also called its \[dq]directory\[dq] ID. 49494 - \[ga]client_id\[ga]: the service principal\[aq]s client ID 49495 - \[ga]client_secret\[ga]: one of the service principal\[aq]s client secrets 49496 49497 The credentials can also be placed in a file using the 49498 \[ga]service_principal_file\[ga] configuration option. 49499 49500 #### Service principal with certificate 49501 49502 If these variables are set, rclone will authenticate with a service principal with certificate. 49503 49504 - \[ga]tenant\[ga]: ID of the service principal\[aq]s tenant. Also called its \[dq]directory\[dq] ID. 49505 - \[ga]client_id\[ga]: the service principal\[aq]s client ID 49506 - \[ga]client_certificate_path\[ga]: path to a PEM or PKCS12 certificate file including the private key. 49507 - \[ga]client_certificate_password\[ga]: (optional) password for the certificate file. 49508 - \[ga]client_send_certificate_chain\[ga]: (optional) Specifies whether an authentication request will include an x5c header to support subject name / issuer based authentication. When set to \[dq]true\[dq] or \[dq]1\[dq], authentication requests include the x5c header. 49509 49510 **NB** \[ga]client_certificate_password\[ga] must be obscured - see [rclone obscure](https://rclone.org/commands/rclone_obscure/). 49511 49512 #### User with username and password 49513 49514 If these variables are set, rclone will authenticate with username and password. 49515 49516 - \[ga]tenant\[ga]: (optional) tenant to authenticate in. Defaults to \[dq]organizations\[dq]. 49517 - \[ga]client_id\[ga]: client ID of the application the user will authenticate to 49518 - \[ga]username\[ga]: a username (usually an email address) 49519 - \[ga]password\[ga]: the user\[aq]s password 49520 49521 Microsoft doesn\[aq]t recommend this kind of authentication, because it\[aq]s 49522 less secure than other authentication flows. This method is not 49523 interactive, so it isn\[aq]t compatible with any form of multi-factor 49524 authentication, and the application must already have user or admin 49525 consent. This credential can only authenticate work and school 49526 accounts; it can\[aq]t authenticate Microsoft accounts. 49527 49528 **NB** \[ga]password\[ga] must be obscured - see [rclone obscure](https://rclone.org/commands/rclone_obscure/). 49529 49530 #### Managed Service Identity Credentials {#use_msi} 49531 49532 If \[ga]use_msi\[ga] is set then managed service identity credentials are 49533 used. This authentication only works when running in an Azure service. 49534 \[ga]env_auth\[ga] needs to be unset to use this. 49535 49536 However if you have multiple user identities to choose from these must 49537 be explicitly specified using exactly one of the \[ga]msi_object_id\[ga], 49538 \[ga]msi_client_id\[ga], or \[ga]msi_mi_res_id\[ga] parameters. 49539 49540 If none of \[ga]msi_object_id\[ga], \[ga]msi_client_id\[ga], or \[ga]msi_mi_res_id\[ga] is 49541 set, this is is equivalent to using \[ga]env_auth\[ga]. 49542 49543 49544 ### Standard options 49545 49546 Here are the Standard options specific to azureblob (Microsoft Azure Blob Storage). 49547 49548 #### --azureblob-account 49549 49550 Azure Storage Account Name. 49551 49552 Set this to the Azure Storage Account Name in use. 49553 49554 Leave blank to use SAS URL or Emulator, otherwise it needs to be set. 49555 49556 If this is blank and if env_auth is set it will be read from the 49557 environment variable \[ga]AZURE_STORAGE_ACCOUNT_NAME\[ga] if possible. 49558 49559 49560 Properties: 49561 49562 - Config: account 49563 - Env Var: RCLONE_AZUREBLOB_ACCOUNT 49564 - Type: string 49565 - Required: false 49566 49567 #### --azureblob-env-auth 49568 49569 Read credentials from runtime (environment variables, CLI or MSI). 49570 49571 See the [authentication docs](/azureblob#authentication) for full info. 49572 49573 Properties: 49574 49575 - Config: env_auth 49576 - Env Var: RCLONE_AZUREBLOB_ENV_AUTH 49577 - Type: bool 49578 - Default: false 49579 49580 #### --azureblob-key 49581 49582 Storage Account Shared Key. 49583 49584 Leave blank to use SAS URL or Emulator. 49585 49586 Properties: 49587 49588 - Config: key 49589 - Env Var: RCLONE_AZUREBLOB_KEY 49590 - Type: string 49591 - Required: false 49592 49593 #### --azureblob-sas-url 49594 49595 SAS URL for container level access only. 49596 49597 Leave blank if using account/key or Emulator. 49598 49599 Properties: 49600 49601 - Config: sas_url 49602 - Env Var: RCLONE_AZUREBLOB_SAS_URL 49603 - Type: string 49604 - Required: false 49605 49606 #### --azureblob-tenant 49607 49608 ID of the service principal\[aq]s tenant. Also called its directory ID. 49609 49610 Set this if using 49611 - Service principal with client secret 49612 - Service principal with certificate 49613 - User with username and password 49614 49615 49616 Properties: 49617 49618 - Config: tenant 49619 - Env Var: RCLONE_AZUREBLOB_TENANT 49620 - Type: string 49621 - Required: false 49622 49623 #### --azureblob-client-id 49624 49625 The ID of the client in use. 49626 49627 Set this if using 49628 - Service principal with client secret 49629 - Service principal with certificate 49630 - User with username and password 49631 49632 49633 Properties: 49634 49635 - Config: client_id 49636 - Env Var: RCLONE_AZUREBLOB_CLIENT_ID 49637 - Type: string 49638 - Required: false 49639 49640 #### --azureblob-client-secret 49641 49642 One of the service principal\[aq]s client secrets 49643 49644 Set this if using 49645 - Service principal with client secret 49646 49647 49648 Properties: 49649 49650 - Config: client_secret 49651 - Env Var: RCLONE_AZUREBLOB_CLIENT_SECRET 49652 - Type: string 49653 - Required: false 49654 49655 #### --azureblob-client-certificate-path 49656 49657 Path to a PEM or PKCS12 certificate file including the private key. 49658 49659 Set this if using 49660 - Service principal with certificate 49661 49662 49663 Properties: 49664 49665 - Config: client_certificate_path 49666 - Env Var: RCLONE_AZUREBLOB_CLIENT_CERTIFICATE_PATH 49667 - Type: string 49668 - Required: false 49669 49670 #### --azureblob-client-certificate-password 49671 49672 Password for the certificate file (optional). 49673 49674 Optionally set this if using 49675 - Service principal with certificate 49676 49677 And the certificate has a password. 49678 49679 49680 **NB** Input to this must be obscured - see [rclone obscure](https://rclone.org/commands/rclone_obscure/). 49681 49682 Properties: 49683 49684 - Config: client_certificate_password 49685 - Env Var: RCLONE_AZUREBLOB_CLIENT_CERTIFICATE_PASSWORD 49686 - Type: string 49687 - Required: false 49688 49689 ### Advanced options 49690 49691 Here are the Advanced options specific to azureblob (Microsoft Azure Blob Storage). 49692 49693 #### --azureblob-client-send-certificate-chain 49694 49695 Send the certificate chain when using certificate auth. 49696 49697 Specifies whether an authentication request will include an x5c header 49698 to support subject name / issuer based authentication. When set to 49699 true, authentication requests include the x5c header. 49700 49701 Optionally set this if using 49702 - Service principal with certificate 49703 49704 49705 Properties: 49706 49707 - Config: client_send_certificate_chain 49708 - Env Var: RCLONE_AZUREBLOB_CLIENT_SEND_CERTIFICATE_CHAIN 49709 - Type: bool 49710 - Default: false 49711 49712 #### --azureblob-username 49713 49714 User name (usually an email address) 49715 49716 Set this if using 49717 - User with username and password 49718 49719 49720 Properties: 49721 49722 - Config: username 49723 - Env Var: RCLONE_AZUREBLOB_USERNAME 49724 - Type: string 49725 - Required: false 49726 49727 #### --azureblob-password 49728 49729 The user\[aq]s password 49730 49731 Set this if using 49732 - User with username and password 49733 49734 49735 **NB** Input to this must be obscured - see [rclone obscure](https://rclone.org/commands/rclone_obscure/). 49736 49737 Properties: 49738 49739 - Config: password 49740 - Env Var: RCLONE_AZUREBLOB_PASSWORD 49741 - Type: string 49742 - Required: false 49743 49744 #### --azureblob-service-principal-file 49745 49746 Path to file containing credentials for use with a service principal. 49747 49748 Leave blank normally. Needed only if you want to use a service principal instead of interactive login. 49749 49750 $ az ad sp create-for-rbac --name \[dq]<name>\[dq] \[rs] 49751 --role \[dq]Storage Blob Data Owner\[dq] \[rs] 49752 --scopes \[dq]/subscriptions/<subscription>/resourceGroups/<resource-group>/providers/Microsoft.Storage/storageAccounts/<storage-account>/blobServices/default/containers/<container>\[dq] \[rs] 49753 > azure-principal.json 49754 49755 See [\[dq]Create an Azure service principal\[dq]](https://docs.microsoft.com/en-us/cli/azure/create-an-azure-service-principal-azure-cli) and [\[dq]Assign an Azure role for access to blob data\[dq]](https://docs.microsoft.com/en-us/azure/storage/common/storage-auth-aad-rbac-cli) pages for more details. 49756 49757 It may be more convenient to put the credentials directly into the 49758 rclone config file under the \[ga]client_id\[ga], \[ga]tenant\[ga] and \[ga]client_secret\[ga] 49759 keys instead of setting \[ga]service_principal_file\[ga]. 49760 49761 49762 Properties: 49763 49764 - Config: service_principal_file 49765 - Env Var: RCLONE_AZUREBLOB_SERVICE_PRINCIPAL_FILE 49766 - Type: string 49767 - Required: false 49768 49769 #### --azureblob-use-msi 49770 49771 Use a managed service identity to authenticate (only works in Azure). 49772 49773 When true, use a [managed service identity](https://docs.microsoft.com/en-us/azure/active-directory/managed-identities-azure-resources/) 49774 to authenticate to Azure Storage instead of a SAS token or account key. 49775 49776 If the VM(SS) on which this program is running has a system-assigned identity, it will 49777 be used by default. If the resource has no system-assigned but exactly one user-assigned identity, 49778 the user-assigned identity will be used by default. If the resource has multiple user-assigned 49779 identities, the identity to use must be explicitly specified using exactly one of the msi_object_id, 49780 msi_client_id, or msi_mi_res_id parameters. 49781 49782 Properties: 49783 49784 - Config: use_msi 49785 - Env Var: RCLONE_AZUREBLOB_USE_MSI 49786 - Type: bool 49787 - Default: false 49788 49789 #### --azureblob-msi-object-id 49790 49791 Object ID of the user-assigned MSI to use, if any. 49792 49793 Leave blank if msi_client_id or msi_mi_res_id specified. 49794 49795 Properties: 49796 49797 - Config: msi_object_id 49798 - Env Var: RCLONE_AZUREBLOB_MSI_OBJECT_ID 49799 - Type: string 49800 - Required: false 49801 49802 #### --azureblob-msi-client-id 49803 49804 Object ID of the user-assigned MSI to use, if any. 49805 49806 Leave blank if msi_object_id or msi_mi_res_id specified. 49807 49808 Properties: 49809 49810 - Config: msi_client_id 49811 - Env Var: RCLONE_AZUREBLOB_MSI_CLIENT_ID 49812 - Type: string 49813 - Required: false 49814 49815 #### --azureblob-msi-mi-res-id 49816 49817 Azure resource ID of the user-assigned MSI to use, if any. 49818 49819 Leave blank if msi_client_id or msi_object_id specified. 49820 49821 Properties: 49822 49823 - Config: msi_mi_res_id 49824 - Env Var: RCLONE_AZUREBLOB_MSI_MI_RES_ID 49825 - Type: string 49826 - Required: false 49827 49828 #### --azureblob-use-emulator 49829 49830 Uses local storage emulator if provided as \[aq]true\[aq]. 49831 49832 Leave blank if using real azure storage endpoint. 49833 49834 Properties: 49835 49836 - Config: use_emulator 49837 - Env Var: RCLONE_AZUREBLOB_USE_EMULATOR 49838 - Type: bool 49839 - Default: false 49840 49841 #### --azureblob-endpoint 49842 49843 Endpoint for the service. 49844 49845 Leave blank normally. 49846 49847 Properties: 49848 49849 - Config: endpoint 49850 - Env Var: RCLONE_AZUREBLOB_ENDPOINT 49851 - Type: string 49852 - Required: false 49853 49854 #### --azureblob-upload-cutoff 49855 49856 Cutoff for switching to chunked upload (<= 256 MiB) (deprecated). 49857 49858 Properties: 49859 49860 - Config: upload_cutoff 49861 - Env Var: RCLONE_AZUREBLOB_UPLOAD_CUTOFF 49862 - Type: string 49863 - Required: false 49864 49865 #### --azureblob-chunk-size 49866 49867 Upload chunk size. 49868 49869 Note that this is stored in memory and there may be up to 49870 \[dq]--transfers\[dq] * \[dq]--azureblob-upload-concurrency\[dq] chunks stored at once 49871 in memory. 49872 49873 Properties: 49874 49875 - Config: chunk_size 49876 - Env Var: RCLONE_AZUREBLOB_CHUNK_SIZE 49877 - Type: SizeSuffix 49878 - Default: 4Mi 49879 49880 #### --azureblob-upload-concurrency 49881 49882 Concurrency for multipart uploads. 49883 49884 This is the number of chunks of the same file that are uploaded 49885 concurrently. 49886 49887 If you are uploading small numbers of large files over high-speed 49888 links and these uploads do not fully utilize your bandwidth, then 49889 increasing this may help to speed up the transfers. 49890 49891 In tests, upload speed increases almost linearly with upload 49892 concurrency. For example to fill a gigabit pipe it may be necessary to 49893 raise this to 64. Note that this will use more memory. 49894 49895 Note that chunks are stored in memory and there may be up to 49896 \[dq]--transfers\[dq] * \[dq]--azureblob-upload-concurrency\[dq] chunks stored at once 49897 in memory. 49898 49899 Properties: 49900 49901 - Config: upload_concurrency 49902 - Env Var: RCLONE_AZUREBLOB_UPLOAD_CONCURRENCY 49903 - Type: int 49904 - Default: 16 49905 49906 #### --azureblob-list-chunk 49907 49908 Size of blob list. 49909 49910 This sets the number of blobs requested in each listing chunk. Default 49911 is the maximum, 5000. \[dq]List blobs\[dq] requests are permitted 2 minutes 49912 per megabyte to complete. If an operation is taking longer than 2 49913 minutes per megabyte on average, it will time out ( 49914 [source](https://docs.microsoft.com/en-us/rest/api/storageservices/setting-timeouts-for-blob-service-operations#exceptions-to-default-timeout-interval) 49915 ). This can be used to limit the number of blobs items to return, to 49916 avoid the time out. 49917 49918 Properties: 49919 49920 - Config: list_chunk 49921 - Env Var: RCLONE_AZUREBLOB_LIST_CHUNK 49922 - Type: int 49923 - Default: 5000 49924 49925 #### --azureblob-access-tier 49926 49927 Access tier of blob: hot, cool, cold or archive. 49928 49929 Archived blobs can be restored by setting access tier to hot, cool or 49930 cold. Leave blank if you intend to use default access tier, which is 49931 set at account level 49932 49933 If there is no \[dq]access tier\[dq] specified, rclone doesn\[aq]t apply any tier. 49934 rclone performs \[dq]Set Tier\[dq] operation on blobs while uploading, if objects 49935 are not modified, specifying \[dq]access tier\[dq] to new one will have no effect. 49936 If blobs are in \[dq]archive tier\[dq] at remote, trying to perform data transfer 49937 operations from remote will not be allowed. User should first restore by 49938 tiering blob to \[dq]Hot\[dq], \[dq]Cool\[dq] or \[dq]Cold\[dq]. 49939 49940 Properties: 49941 49942 - Config: access_tier 49943 - Env Var: RCLONE_AZUREBLOB_ACCESS_TIER 49944 - Type: string 49945 - Required: false 49946 49947 #### --azureblob-archive-tier-delete 49948 49949 Delete archive tier blobs before overwriting. 49950 49951 Archive tier blobs cannot be updated. So without this flag, if you 49952 attempt to update an archive tier blob, then rclone will produce the 49953 error: 49954 49955 can\[aq]t update archive tier blob without --azureblob-archive-tier-delete 49956 49957 With this flag set then before rclone attempts to overwrite an archive 49958 tier blob, it will delete the existing blob before uploading its 49959 replacement. This has the potential for data loss if the upload fails 49960 (unlike updating a normal blob) and also may cost more since deleting 49961 archive tier blobs early may be chargable. 49962 49963 49964 Properties: 49965 49966 - Config: archive_tier_delete 49967 - Env Var: RCLONE_AZUREBLOB_ARCHIVE_TIER_DELETE 49968 - Type: bool 49969 - Default: false 49970 49971 #### --azureblob-disable-checksum 49972 49973 Don\[aq]t store MD5 checksum with object metadata. 49974 49975 Normally rclone will calculate the MD5 checksum of the input before 49976 uploading it so it can add it to metadata on the object. This is great 49977 for data integrity checking but can cause long delays for large files 49978 to start uploading. 49979 49980 Properties: 49981 49982 - Config: disable_checksum 49983 - Env Var: RCLONE_AZUREBLOB_DISABLE_CHECKSUM 49984 - Type: bool 49985 - Default: false 49986 49987 #### --azureblob-memory-pool-flush-time 49988 49989 How often internal memory buffer pools will be flushed. (no longer used) 49990 49991 Properties: 49992 49993 - Config: memory_pool_flush_time 49994 - Env Var: RCLONE_AZUREBLOB_MEMORY_POOL_FLUSH_TIME 49995 - Type: Duration 49996 - Default: 1m0s 49997 49998 #### --azureblob-memory-pool-use-mmap 49999 50000 Whether to use mmap buffers in internal memory pool. (no longer used) 50001 50002 Properties: 50003 50004 - Config: memory_pool_use_mmap 50005 - Env Var: RCLONE_AZUREBLOB_MEMORY_POOL_USE_MMAP 50006 - Type: bool 50007 - Default: false 50008 50009 #### --azureblob-encoding 50010 50011 The encoding for the backend. 50012 50013 See the [encoding section in the overview](https://rclone.org/overview/#encoding) for more info. 50014 50015 Properties: 50016 50017 - Config: encoding 50018 - Env Var: RCLONE_AZUREBLOB_ENCODING 50019 - Type: Encoding 50020 - Default: Slash,BackSlash,Del,Ctl,RightPeriod,InvalidUtf8 50021 50022 #### --azureblob-public-access 50023 50024 Public access level of a container: blob or container. 50025 50026 Properties: 50027 50028 - Config: public_access 50029 - Env Var: RCLONE_AZUREBLOB_PUBLIC_ACCESS 50030 - Type: string 50031 - Required: false 50032 - Examples: 50033 - \[dq]\[dq] 50034 - The container and its blobs can be accessed only with an authorized request. 50035 - It\[aq]s a default value. 50036 - \[dq]blob\[dq] 50037 - Blob data within this container can be read via anonymous request. 50038 - \[dq]container\[dq] 50039 - Allow full public read access for container and blob data. 50040 50041 #### --azureblob-directory-markers 50042 50043 Upload an empty object with a trailing slash when a new directory is created 50044 50045 Empty folders are unsupported for bucket based remotes, this option 50046 creates an empty object ending with \[dq]/\[dq], to persist the folder. 50047 50048 This object also has the metadata \[dq]hdi_isfolder = true\[dq] to conform to 50049 the Microsoft standard. 50050 50051 50052 Properties: 50053 50054 - Config: directory_markers 50055 - Env Var: RCLONE_AZUREBLOB_DIRECTORY_MARKERS 50056 - Type: bool 50057 - Default: false 50058 50059 #### --azureblob-no-check-container 50060 50061 If set, don\[aq]t attempt to check the container exists or create it. 50062 50063 This can be useful when trying to minimise the number of transactions 50064 rclone does if you know the container exists already. 50065 50066 50067 Properties: 50068 50069 - Config: no_check_container 50070 - Env Var: RCLONE_AZUREBLOB_NO_CHECK_CONTAINER 50071 - Type: bool 50072 - Default: false 50073 50074 #### --azureblob-no-head-object 50075 50076 If set, do not do HEAD before GET when getting objects. 50077 50078 Properties: 50079 50080 - Config: no_head_object 50081 - Env Var: RCLONE_AZUREBLOB_NO_HEAD_OBJECT 50082 - Type: bool 50083 - Default: false 50084 50085 #### --azureblob-delete-snapshots 50086 50087 Set to specify how to deal with snapshots on blob deletion. 50088 50089 Properties: 50090 50091 - Config: delete_snapshots 50092 - Env Var: RCLONE_AZUREBLOB_DELETE_SNAPSHOTS 50093 - Type: string 50094 - Required: false 50095 - Choices: 50096 - \[dq]\[dq] 50097 - By default, the delete operation fails if a blob has snapshots 50098 - \[dq]include\[dq] 50099 - Specify \[aq]include\[aq] to remove the root blob and all its snapshots 50100 - \[dq]only\[dq] 50101 - Specify \[aq]only\[aq] to remove only the snapshots but keep the root blob. 50102 50103 #### --azureblob-description 50104 50105 Description of the remote 50106 50107 Properties: 50108 50109 - Config: description 50110 - Env Var: RCLONE_AZUREBLOB_DESCRIPTION 50111 - Type: string 50112 - Required: false 50113 50114 50115 50116 ### Custom upload headers 50117 50118 You can set custom upload headers with the \[ga]--header-upload\[ga] flag. 50119 50120 - Cache-Control 50121 - Content-Disposition 50122 - Content-Encoding 50123 - Content-Language 50124 - Content-Type 50125 50126 Eg \[ga]--header-upload \[dq]Content-Type: text/potato\[dq]\[ga] 50127 50128 ## Limitations 50129 50130 MD5 sums are only uploaded with chunked files if the source has an MD5 50131 sum. This will always be the case for a local to azure copy. 50132 50133 \[ga]rclone about\[ga] is not supported by the Microsoft Azure Blob storage backend. Backends without 50134 this capability cannot determine free space for an rclone mount or 50135 use policy \[ga]mfs\[ga] (most free space) as a member of an rclone union 50136 remote. 50137 50138 See [List of backends that do not support rclone about](https://rclone.org/overview/#optional-features) and [rclone about](https://rclone.org/commands/rclone_about/) 50139 50140 ## Azure Storage Emulator Support 50141 50142 You can run rclone with the storage emulator (usually _azurite_). 50143 50144 To do this, just set up a new remote with \[ga]rclone config\[ga] following 50145 the instructions in the introduction and set \[ga]use_emulator\[ga] in the 50146 advanced settings as \[ga]true\[ga]. You do not need to provide a default 50147 account name nor an account key. But you can override them in the 50148 \[ga]account\[ga] and \[ga]key\[ga] options. (Prior to v1.61 they were hard coded to 50149 _azurite_\[aq]s \[ga]devstoreaccount1\[ga].) 50150 50151 Also, if you want to access a storage emulator instance running on a 50152 different machine, you can override the \[ga]endpoint\[ga] parameter in the 50153 advanced settings, setting it to 50154 \[ga]http(s)://<host>:<port>/devstoreaccount1\[ga] 50155 (e.g. \[ga]http://10.254.2.5:10000/devstoreaccount1\[ga]). 50156 50157 # Microsoft Azure Files Storage 50158 50159 Paths are specified as \[ga]remote:\[ga] You may put subdirectories in too, 50160 e.g. \[ga]remote:path/to/dir\[ga]. 50161 50162 ## Configuration 50163 50164 Here is an example of making a Microsoft Azure Files Storage 50165 configuration. For a remote called \[ga]remote\[ga]. First run: 50166 50167 rclone config 50168 50169 This will guide you through an interactive setup process: 50170 \f[R] 50171 .fi 50172 .PP 50173 No remotes found, make a new one? 50174 n) New remote s) Set configuration password q) Quit config n/s/q> n 50175 name> remote Type of storage to configure. 50176 Choose a number from below, or type in your own value [snip] XX / 50177 Microsoft Azure Files Storage \ \[dq]azurefiles\[dq] [snip] 50178 .PP 50179 Option account. 50180 Azure Storage Account Name. 50181 Set this to the Azure Storage Account Name in use. 50182 Leave blank to use SAS URL or connection string, otherwise it needs to 50183 be set. 50184 If this is blank and if env_auth is set it will be read from the 50185 environment variable \f[C]AZURE_STORAGE_ACCOUNT_NAME\f[R] if possible. 50186 Enter a value. 50187 Press Enter to leave empty. 50188 account> account_name 50189 .PP 50190 Option share_name. 50191 Azure Files Share Name. 50192 This is required and is the name of the share to access. 50193 Enter a value. 50194 Press Enter to leave empty. 50195 share_name> share_name 50196 .PP 50197 Option env_auth. 50198 Read credentials from runtime (environment variables, CLI or MSI). 50199 See the authentication docs for full info. 50200 Enter a boolean value (true or false). 50201 Press Enter for the default (false). 50202 env_auth> 50203 .PP 50204 Option key. 50205 Storage Account Shared Key. 50206 Leave blank to use SAS URL or connection string. 50207 Enter a value. 50208 Press Enter to leave empty. 50209 key> base64encodedkey== 50210 .PP 50211 Option sas_url. 50212 SAS URL. 50213 Leave blank if using account/key or connection string. 50214 Enter a value. 50215 Press Enter to leave empty. 50216 sas_url> 50217 .PP 50218 Option connection_string. 50219 Azure Files Connection String. 50220 Enter a value. 50221 Press Enter to leave empty. 50222 connection_string> [snip] 50223 .PP 50224 Configuration complete. 50225 Options: - type: azurefiles - account: account_name - share_name: 50226 share_name - key: base64encodedkey== Keep this \[dq]remote\[dq] remote? 50227 y) Yes this is OK (default) e) Edit this remote d) Delete this remote 50228 y/e/d> 50229 .IP 50230 .nf 50231 \f[C] 50232 Once configured you can use rclone. 50233 50234 See all files in the top level: 50235 50236 rclone lsf remote: 50237 50238 Make a new directory in the root: 50239 50240 rclone mkdir remote:dir 50241 50242 Recursively List the contents: 50243 50244 rclone ls remote: 50245 50246 Sync \[ga]/home/local/directory\[ga] to the remote directory, deleting any 50247 excess files in the directory. 50248 50249 rclone sync --interactive /home/local/directory remote:dir 50250 50251 ### Modified time 50252 50253 The modified time is stored as Azure standard \[ga]LastModified\[ga] time on 50254 files 50255 50256 ### Performance 50257 50258 When uploading large files, increasing the value of 50259 \[ga]--azurefiles-upload-concurrency\[ga] will increase performance at the cost 50260 of using more memory. The default of 16 is set quite conservatively to 50261 use less memory. It maybe be necessary raise it to 64 or higher to 50262 fully utilize a 1 GBit/s link with a single file transfer. 50263 50264 ### Restricted filename characters 50265 50266 In addition to the [default restricted characters set](https://rclone.org/overview/#restricted-characters) 50267 the following characters are also replaced: 50268 50269 | Character | Value | Replacement | 50270 | --------- |:-----:|:-----------:| 50271 | \[dq] | 0x22 | \[uFF02] | 50272 | * | 0x2A | \[uFF0A] | 50273 | : | 0x3A | \[uFF1A] | 50274 | < | 0x3C | \[uFF1C] | 50275 | > | 0x3E | \[uFF1E] | 50276 | ? | 0x3F | \[uFF1F] | 50277 | \[rs] | 0x5C | \[uFF3C] | 50278 | \[rs]| | 0x7C | \[uFF5C] | 50279 50280 File names can also not end with the following characters. 50281 These only get replaced if they are the last character in the name: 50282 50283 | Character | Value | Replacement | 50284 | --------- |:-----:|:-----------:| 50285 | . | 0x2E | \[uFF0E] | 50286 50287 Invalid UTF-8 bytes will also be [replaced](https://rclone.org/overview/#invalid-utf8), 50288 as they can\[aq]t be used in JSON strings. 50289 50290 ### Hashes 50291 50292 MD5 hashes are stored with files. Not all files will have MD5 hashes 50293 as these have to be uploaded with the file. 50294 50295 ### Authentication {#authentication} 50296 50297 There are a number of ways of supplying credentials for Azure Files 50298 Storage. Rclone tries them in the order of the sections below. 50299 50300 #### Env Auth 50301 50302 If the \[ga]env_auth\[ga] config parameter is \[ga]true\[ga] then rclone will pull 50303 credentials from the environment or runtime. 50304 50305 It tries these authentication methods in this order: 50306 50307 1. Environment Variables 50308 2. Managed Service Identity Credentials 50309 3. Azure CLI credentials (as used by the az tool) 50310 50311 These are described in the following sections 50312 50313 ##### Env Auth: 1. Environment Variables 50314 50315 If \[ga]env_auth\[ga] is set and environment variables are present rclone 50316 authenticates a service principal with a secret or certificate, or a 50317 user with a password, depending on which environment variable are set. 50318 It reads configuration from these variables, in the following order: 50319 50320 1. Service principal with client secret 50321 - \[ga]AZURE_TENANT_ID\[ga]: ID of the service principal\[aq]s tenant. Also called its \[dq]directory\[dq] ID. 50322 - \[ga]AZURE_CLIENT_ID\[ga]: the service principal\[aq]s client ID 50323 - \[ga]AZURE_CLIENT_SECRET\[ga]: one of the service principal\[aq]s client secrets 50324 2. Service principal with certificate 50325 - \[ga]AZURE_TENANT_ID\[ga]: ID of the service principal\[aq]s tenant. Also called its \[dq]directory\[dq] ID. 50326 - \[ga]AZURE_CLIENT_ID\[ga]: the service principal\[aq]s client ID 50327 - \[ga]AZURE_CLIENT_CERTIFICATE_PATH\[ga]: path to a PEM or PKCS12 certificate file including the private key. 50328 - \[ga]AZURE_CLIENT_CERTIFICATE_PASSWORD\[ga]: (optional) password for the certificate file. 50329 - \[ga]AZURE_CLIENT_SEND_CERTIFICATE_CHAIN\[ga]: (optional) Specifies whether an authentication request will include an x5c header to support subject name / issuer based authentication. When set to \[dq]true\[dq] or \[dq]1\[dq], authentication requests include the x5c header. 50330 3. User with username and password 50331 - \[ga]AZURE_TENANT_ID\[ga]: (optional) tenant to authenticate in. Defaults to \[dq]organizations\[dq]. 50332 - \[ga]AZURE_CLIENT_ID\[ga]: client ID of the application the user will authenticate to 50333 - \[ga]AZURE_USERNAME\[ga]: a username (usually an email address) 50334 - \[ga]AZURE_PASSWORD\[ga]: the user\[aq]s password 50335 4. Workload Identity 50336 - \[ga]AZURE_TENANT_ID\[ga]: Tenant to authenticate in. 50337 - \[ga]AZURE_CLIENT_ID\[ga]: Client ID of the application the user will authenticate to. 50338 - \[ga]AZURE_FEDERATED_TOKEN_FILE\[ga]: Path to projected service account token file. 50339 - \[ga]AZURE_AUTHORITY_HOST\[ga]: Authority of an Azure Active Directory endpoint (default: login.microsoftonline.com). 50340 50341 50342 ##### Env Auth: 2. Managed Service Identity Credentials 50343 50344 When using Managed Service Identity if the VM(SS) on which this 50345 program is running has a system-assigned identity, it will be used by 50346 default. If the resource has no system-assigned but exactly one 50347 user-assigned identity, the user-assigned identity will be used by 50348 default. 50349 50350 If the resource has multiple user-assigned identities you will need to 50351 unset \[ga]env_auth\[ga] and set \[ga]use_msi\[ga] instead. See the [\[ga]use_msi\[ga] 50352 section](#use_msi). 50353 50354 ##### Env Auth: 3. Azure CLI credentials (as used by the az tool) 50355 50356 Credentials created with the \[ga]az\[ga] tool can be picked up using \[ga]env_auth\[ga]. 50357 50358 For example if you were to login with a service principal like this: 50359 50360 az login --service-principal -u XXX -p XXX --tenant XXX 50361 50362 Then you could access rclone resources like this: 50363 50364 rclone lsf :azurefiles,env_auth,account=ACCOUNT: 50365 50366 Or 50367 50368 rclone lsf --azurefiles-env-auth --azurefiles-account=ACCOUNT :azurefiles: 50369 50370 #### Account and Shared Key 50371 50372 This is the most straight forward and least flexible way. Just fill 50373 in the \[ga]account\[ga] and \[ga]key\[ga] lines and leave the rest blank. 50374 50375 #### SAS URL 50376 50377 To use it leave \[ga]account\[ga], \[ga]key\[ga] and \[ga]connection_string\[ga] blank and fill in \[ga]sas_url\[ga]. 50378 50379 #### Connection String 50380 50381 To use it leave \[ga]account\[ga], \[ga]key\[ga] and \[dq]sas_url\[dq] blank and fill in \[ga]connection_string\[ga]. 50382 50383 #### Service principal with client secret 50384 50385 If these variables are set, rclone will authenticate with a service principal with a client secret. 50386 50387 - \[ga]tenant\[ga]: ID of the service principal\[aq]s tenant. Also called its \[dq]directory\[dq] ID. 50388 - \[ga]client_id\[ga]: the service principal\[aq]s client ID 50389 - \[ga]client_secret\[ga]: one of the service principal\[aq]s client secrets 50390 50391 The credentials can also be placed in a file using the 50392 \[ga]service_principal_file\[ga] configuration option. 50393 50394 #### Service principal with certificate 50395 50396 If these variables are set, rclone will authenticate with a service principal with certificate. 50397 50398 - \[ga]tenant\[ga]: ID of the service principal\[aq]s tenant. Also called its \[dq]directory\[dq] ID. 50399 - \[ga]client_id\[ga]: the service principal\[aq]s client ID 50400 - \[ga]client_certificate_path\[ga]: path to a PEM or PKCS12 certificate file including the private key. 50401 - \[ga]client_certificate_password\[ga]: (optional) password for the certificate file. 50402 - \[ga]client_send_certificate_chain\[ga]: (optional) Specifies whether an authentication request will include an x5c header to support subject name / issuer based authentication. When set to \[dq]true\[dq] or \[dq]1\[dq], authentication requests include the x5c header. 50403 50404 **NB** \[ga]client_certificate_password\[ga] must be obscured - see [rclone obscure](https://rclone.org/commands/rclone_obscure/). 50405 50406 #### User with username and password 50407 50408 If these variables are set, rclone will authenticate with username and password. 50409 50410 - \[ga]tenant\[ga]: (optional) tenant to authenticate in. Defaults to \[dq]organizations\[dq]. 50411 - \[ga]client_id\[ga]: client ID of the application the user will authenticate to 50412 - \[ga]username\[ga]: a username (usually an email address) 50413 - \[ga]password\[ga]: the user\[aq]s password 50414 50415 Microsoft doesn\[aq]t recommend this kind of authentication, because it\[aq]s 50416 less secure than other authentication flows. This method is not 50417 interactive, so it isn\[aq]t compatible with any form of multi-factor 50418 authentication, and the application must already have user or admin 50419 consent. This credential can only authenticate work and school 50420 accounts; it can\[aq]t authenticate Microsoft accounts. 50421 50422 **NB** \[ga]password\[ga] must be obscured - see [rclone obscure](https://rclone.org/commands/rclone_obscure/). 50423 50424 #### Managed Service Identity Credentials {#use_msi} 50425 50426 If \[ga]use_msi\[ga] is set then managed service identity credentials are 50427 used. This authentication only works when running in an Azure service. 50428 \[ga]env_auth\[ga] needs to be unset to use this. 50429 50430 However if you have multiple user identities to choose from these must 50431 be explicitly specified using exactly one of the \[ga]msi_object_id\[ga], 50432 \[ga]msi_client_id\[ga], or \[ga]msi_mi_res_id\[ga] parameters. 50433 50434 If none of \[ga]msi_object_id\[ga], \[ga]msi_client_id\[ga], or \[ga]msi_mi_res_id\[ga] is 50435 set, this is is equivalent to using \[ga]env_auth\[ga]. 50436 50437 50438 ### Standard options 50439 50440 Here are the Standard options specific to azurefiles (Microsoft Azure Files). 50441 50442 #### --azurefiles-account 50443 50444 Azure Storage Account Name. 50445 50446 Set this to the Azure Storage Account Name in use. 50447 50448 Leave blank to use SAS URL or connection string, otherwise it needs to be set. 50449 50450 If this is blank and if env_auth is set it will be read from the 50451 environment variable \[ga]AZURE_STORAGE_ACCOUNT_NAME\[ga] if possible. 50452 50453 50454 Properties: 50455 50456 - Config: account 50457 - Env Var: RCLONE_AZUREFILES_ACCOUNT 50458 - Type: string 50459 - Required: false 50460 50461 #### --azurefiles-share-name 50462 50463 Azure Files Share Name. 50464 50465 This is required and is the name of the share to access. 50466 50467 50468 Properties: 50469 50470 - Config: share_name 50471 - Env Var: RCLONE_AZUREFILES_SHARE_NAME 50472 - Type: string 50473 - Required: false 50474 50475 #### --azurefiles-env-auth 50476 50477 Read credentials from runtime (environment variables, CLI or MSI). 50478 50479 See the [authentication docs](/azurefiles#authentication) for full info. 50480 50481 Properties: 50482 50483 - Config: env_auth 50484 - Env Var: RCLONE_AZUREFILES_ENV_AUTH 50485 - Type: bool 50486 - Default: false 50487 50488 #### --azurefiles-key 50489 50490 Storage Account Shared Key. 50491 50492 Leave blank to use SAS URL or connection string. 50493 50494 Properties: 50495 50496 - Config: key 50497 - Env Var: RCLONE_AZUREFILES_KEY 50498 - Type: string 50499 - Required: false 50500 50501 #### --azurefiles-sas-url 50502 50503 SAS URL. 50504 50505 Leave blank if using account/key or connection string. 50506 50507 Properties: 50508 50509 - Config: sas_url 50510 - Env Var: RCLONE_AZUREFILES_SAS_URL 50511 - Type: string 50512 - Required: false 50513 50514 #### --azurefiles-connection-string 50515 50516 Azure Files Connection String. 50517 50518 Properties: 50519 50520 - Config: connection_string 50521 - Env Var: RCLONE_AZUREFILES_CONNECTION_STRING 50522 - Type: string 50523 - Required: false 50524 50525 #### --azurefiles-tenant 50526 50527 ID of the service principal\[aq]s tenant. Also called its directory ID. 50528 50529 Set this if using 50530 - Service principal with client secret 50531 - Service principal with certificate 50532 - User with username and password 50533 50534 50535 Properties: 50536 50537 - Config: tenant 50538 - Env Var: RCLONE_AZUREFILES_TENANT 50539 - Type: string 50540 - Required: false 50541 50542 #### --azurefiles-client-id 50543 50544 The ID of the client in use. 50545 50546 Set this if using 50547 - Service principal with client secret 50548 - Service principal with certificate 50549 - User with username and password 50550 50551 50552 Properties: 50553 50554 - Config: client_id 50555 - Env Var: RCLONE_AZUREFILES_CLIENT_ID 50556 - Type: string 50557 - Required: false 50558 50559 #### --azurefiles-client-secret 50560 50561 One of the service principal\[aq]s client secrets 50562 50563 Set this if using 50564 - Service principal with client secret 50565 50566 50567 Properties: 50568 50569 - Config: client_secret 50570 - Env Var: RCLONE_AZUREFILES_CLIENT_SECRET 50571 - Type: string 50572 - Required: false 50573 50574 #### --azurefiles-client-certificate-path 50575 50576 Path to a PEM or PKCS12 certificate file including the private key. 50577 50578 Set this if using 50579 - Service principal with certificate 50580 50581 50582 Properties: 50583 50584 - Config: client_certificate_path 50585 - Env Var: RCLONE_AZUREFILES_CLIENT_CERTIFICATE_PATH 50586 - Type: string 50587 - Required: false 50588 50589 #### --azurefiles-client-certificate-password 50590 50591 Password for the certificate file (optional). 50592 50593 Optionally set this if using 50594 - Service principal with certificate 50595 50596 And the certificate has a password. 50597 50598 50599 **NB** Input to this must be obscured - see [rclone obscure](https://rclone.org/commands/rclone_obscure/). 50600 50601 Properties: 50602 50603 - Config: client_certificate_password 50604 - Env Var: RCLONE_AZUREFILES_CLIENT_CERTIFICATE_PASSWORD 50605 - Type: string 50606 - Required: false 50607 50608 ### Advanced options 50609 50610 Here are the Advanced options specific to azurefiles (Microsoft Azure Files). 50611 50612 #### --azurefiles-client-send-certificate-chain 50613 50614 Send the certificate chain when using certificate auth. 50615 50616 Specifies whether an authentication request will include an x5c header 50617 to support subject name / issuer based authentication. When set to 50618 true, authentication requests include the x5c header. 50619 50620 Optionally set this if using 50621 - Service principal with certificate 50622 50623 50624 Properties: 50625 50626 - Config: client_send_certificate_chain 50627 - Env Var: RCLONE_AZUREFILES_CLIENT_SEND_CERTIFICATE_CHAIN 50628 - Type: bool 50629 - Default: false 50630 50631 #### --azurefiles-username 50632 50633 User name (usually an email address) 50634 50635 Set this if using 50636 - User with username and password 50637 50638 50639 Properties: 50640 50641 - Config: username 50642 - Env Var: RCLONE_AZUREFILES_USERNAME 50643 - Type: string 50644 - Required: false 50645 50646 #### --azurefiles-password 50647 50648 The user\[aq]s password 50649 50650 Set this if using 50651 - User with username and password 50652 50653 50654 **NB** Input to this must be obscured - see [rclone obscure](https://rclone.org/commands/rclone_obscure/). 50655 50656 Properties: 50657 50658 - Config: password 50659 - Env Var: RCLONE_AZUREFILES_PASSWORD 50660 - Type: string 50661 - Required: false 50662 50663 #### --azurefiles-service-principal-file 50664 50665 Path to file containing credentials for use with a service principal. 50666 50667 Leave blank normally. Needed only if you want to use a service principal instead of interactive login. 50668 50669 $ az ad sp create-for-rbac --name \[dq]<name>\[dq] \[rs] 50670 --role \[dq]Storage Files Data Owner\[dq] \[rs] 50671 --scopes \[dq]/subscriptions/<subscription>/resourceGroups/<resource-group>/providers/Microsoft.Storage/storageAccounts/<storage-account>/blobServices/default/containers/<container>\[dq] \[rs] 50672 > azure-principal.json 50673 50674 See [\[dq]Create an Azure service principal\[dq]](https://docs.microsoft.com/en-us/cli/azure/create-an-azure-service-principal-azure-cli) and [\[dq]Assign an Azure role for access to files data\[dq]](https://docs.microsoft.com/en-us/azure/storage/common/storage-auth-aad-rbac-cli) pages for more details. 50675 50676 **NB** this section needs updating for Azure Files - pull requests appreciated! 50677 50678 It may be more convenient to put the credentials directly into the 50679 rclone config file under the \[ga]client_id\[ga], \[ga]tenant\[ga] and \[ga]client_secret\[ga] 50680 keys instead of setting \[ga]service_principal_file\[ga]. 50681 50682 50683 Properties: 50684 50685 - Config: service_principal_file 50686 - Env Var: RCLONE_AZUREFILES_SERVICE_PRINCIPAL_FILE 50687 - Type: string 50688 - Required: false 50689 50690 #### --azurefiles-use-msi 50691 50692 Use a managed service identity to authenticate (only works in Azure). 50693 50694 When true, use a [managed service identity](https://docs.microsoft.com/en-us/azure/active-directory/managed-identities-azure-resources/) 50695 to authenticate to Azure Storage instead of a SAS token or account key. 50696 50697 If the VM(SS) on which this program is running has a system-assigned identity, it will 50698 be used by default. If the resource has no system-assigned but exactly one user-assigned identity, 50699 the user-assigned identity will be used by default. If the resource has multiple user-assigned 50700 identities, the identity to use must be explicitly specified using exactly one of the msi_object_id, 50701 msi_client_id, or msi_mi_res_id parameters. 50702 50703 Properties: 50704 50705 - Config: use_msi 50706 - Env Var: RCLONE_AZUREFILES_USE_MSI 50707 - Type: bool 50708 - Default: false 50709 50710 #### --azurefiles-msi-object-id 50711 50712 Object ID of the user-assigned MSI to use, if any. 50713 50714 Leave blank if msi_client_id or msi_mi_res_id specified. 50715 50716 Properties: 50717 50718 - Config: msi_object_id 50719 - Env Var: RCLONE_AZUREFILES_MSI_OBJECT_ID 50720 - Type: string 50721 - Required: false 50722 50723 #### --azurefiles-msi-client-id 50724 50725 Object ID of the user-assigned MSI to use, if any. 50726 50727 Leave blank if msi_object_id or msi_mi_res_id specified. 50728 50729 Properties: 50730 50731 - Config: msi_client_id 50732 - Env Var: RCLONE_AZUREFILES_MSI_CLIENT_ID 50733 - Type: string 50734 - Required: false 50735 50736 #### --azurefiles-msi-mi-res-id 50737 50738 Azure resource ID of the user-assigned MSI to use, if any. 50739 50740 Leave blank if msi_client_id or msi_object_id specified. 50741 50742 Properties: 50743 50744 - Config: msi_mi_res_id 50745 - Env Var: RCLONE_AZUREFILES_MSI_MI_RES_ID 50746 - Type: string 50747 - Required: false 50748 50749 #### --azurefiles-endpoint 50750 50751 Endpoint for the service. 50752 50753 Leave blank normally. 50754 50755 Properties: 50756 50757 - Config: endpoint 50758 - Env Var: RCLONE_AZUREFILES_ENDPOINT 50759 - Type: string 50760 - Required: false 50761 50762 #### --azurefiles-chunk-size 50763 50764 Upload chunk size. 50765 50766 Note that this is stored in memory and there may be up to 50767 \[dq]--transfers\[dq] * \[dq]--azurefile-upload-concurrency\[dq] chunks stored at once 50768 in memory. 50769 50770 Properties: 50771 50772 - Config: chunk_size 50773 - Env Var: RCLONE_AZUREFILES_CHUNK_SIZE 50774 - Type: SizeSuffix 50775 - Default: 4Mi 50776 50777 #### --azurefiles-upload-concurrency 50778 50779 Concurrency for multipart uploads. 50780 50781 This is the number of chunks of the same file that are uploaded 50782 concurrently. 50783 50784 If you are uploading small numbers of large files over high-speed 50785 links and these uploads do not fully utilize your bandwidth, then 50786 increasing this may help to speed up the transfers. 50787 50788 Note that chunks are stored in memory and there may be up to 50789 \[dq]--transfers\[dq] * \[dq]--azurefile-upload-concurrency\[dq] chunks stored at once 50790 in memory. 50791 50792 Properties: 50793 50794 - Config: upload_concurrency 50795 - Env Var: RCLONE_AZUREFILES_UPLOAD_CONCURRENCY 50796 - Type: int 50797 - Default: 16 50798 50799 #### --azurefiles-max-stream-size 50800 50801 Max size for streamed files. 50802 50803 Azure files needs to know in advance how big the file will be. When 50804 rclone doesn\[aq]t know it uses this value instead. 50805 50806 This will be used when rclone is streaming data, the most common uses are: 50807 50808 - Uploading files with \[ga]--vfs-cache-mode off\[ga] with \[ga]rclone mount\[ga] 50809 - Using \[ga]rclone rcat\[ga] 50810 - Copying files with unknown length 50811 50812 You will need this much free space in the share as the file will be this size temporarily. 50813 50814 50815 Properties: 50816 50817 - Config: max_stream_size 50818 - Env Var: RCLONE_AZUREFILES_MAX_STREAM_SIZE 50819 - Type: SizeSuffix 50820 - Default: 10Gi 50821 50822 #### --azurefiles-encoding 50823 50824 The encoding for the backend. 50825 50826 See the [encoding section in the overview](https://rclone.org/overview/#encoding) for more info. 50827 50828 Properties: 50829 50830 - Config: encoding 50831 - Env Var: RCLONE_AZUREFILES_ENCODING 50832 - Type: Encoding 50833 - Default: Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Del,Ctl,RightPeriod,InvalidUtf8,Dot 50834 50835 #### --azurefiles-description 50836 50837 Description of the remote 50838 50839 Properties: 50840 50841 - Config: description 50842 - Env Var: RCLONE_AZUREFILES_DESCRIPTION 50843 - Type: string 50844 - Required: false 50845 50846 50847 50848 ### Custom upload headers 50849 50850 You can set custom upload headers with the \[ga]--header-upload\[ga] flag. 50851 50852 - Cache-Control 50853 - Content-Disposition 50854 - Content-Encoding 50855 - Content-Language 50856 - Content-Type 50857 50858 Eg \[ga]--header-upload \[dq]Content-Type: text/potato\[dq]\[ga] 50859 50860 ## Limitations 50861 50862 MD5 sums are only uploaded with chunked files if the source has an MD5 50863 sum. This will always be the case for a local to azure copy. 50864 50865 # Microsoft OneDrive 50866 50867 Paths are specified as \[ga]remote:path\[ga] 50868 50869 Paths may be as deep as required, e.g. \[ga]remote:directory/subdirectory\[ga]. 50870 50871 ## Configuration 50872 50873 The initial setup for OneDrive involves getting a token from 50874 Microsoft which you need to do in your browser. \[ga]rclone config\[ga] walks 50875 you through it. 50876 50877 Here is an example of how to make a remote called \[ga]remote\[ga]. First run: 50878 50879 rclone config 50880 50881 This will guide you through an interactive setup process: 50882 \f[R] 50883 .fi 50884 .IP "e)" 3 50885 Edit existing remote 50886 .IP "f)" 3 50887 New remote 50888 .IP "g)" 3 50889 Delete remote 50890 .IP "h)" 3 50891 Rename remote 50892 .IP "i)" 3 50893 Copy remote 50894 .IP "j)" 3 50895 Set configuration password 50896 .IP "k)" 3 50897 Quit config e/n/d/r/c/s/q> n name> remote Type of storage to configure. 50898 Enter a string value. 50899 Press Enter for the default (\[dq]\[dq]). 50900 Choose a number from below, or type in your own value [snip] XX / 50901 Microsoft OneDrive \ \[dq]onedrive\[dq] [snip] Storage> onedrive 50902 Microsoft App Client Id Leave blank normally. 50903 Enter a string value. 50904 Press Enter for the default (\[dq]\[dq]). 50905 client_id> Microsoft App Client Secret Leave blank normally. 50906 Enter a string value. 50907 Press Enter for the default (\[dq]\[dq]). 50908 client_secret> Edit advanced config? 50909 (y/n) 50910 .IP "l)" 3 50911 Yes 50912 .IP "m)" 3 50913 No y/n> n Remote config Use web browser to automatically authenticate 50914 rclone with remote? 50915 .IP \[bu] 2 50916 Say Y if the machine running rclone has a web browser you can use 50917 .IP \[bu] 2 50918 Say N if running rclone on a (remote) machine without web browser access 50919 If not sure try Y. 50920 If Y failed, try N. 50921 .IP "y)" 3 50922 Yes 50923 .IP "z)" 3 50924 No y/n> y If your browser doesn\[aq]t open automatically go to the 50925 following link: http://127.0.0.1:53682/auth Log in and authorize rclone 50926 for access Waiting for code... 50927 Got code Choose a number from below, or type in an existing value 1 / 50928 OneDrive Personal or Business \ \[dq]onedrive\[dq] 2 / Sharepoint site 50929 \ \[dq]sharepoint\[dq] 3 / Type in driveID \ \[dq]driveid\[dq] 4 / Type 50930 in SiteID \ \[dq]siteid\[dq] 5 / Search a Sharepoint site 50931 \ \[dq]search\[dq] Your choice> 1 Found 1 drives, please select the one 50932 you want to use: 0: OneDrive (business) 50933 id=b!Eqwertyuiopasdfghjklzxcvbnm-7mnbvcxzlkjhgfdsapoiuytrewqk Chose 50934 drive to use:> 0 Found drive \[aq]root\[aq] of type \[aq]business\[aq], 50935 URL: https://org-my.sharepoint.com/personal/you/Documents Is that okay? 50936 .IP "a)" 3 50937 Yes 50938 .IP "b)" 3 50939 No y/n> y -------------------- [remote] type = onedrive token = 50940 {\[dq]access_token\[dq]:\[dq]youraccesstoken\[dq],\[dq]token_type\[dq]:\[dq]Bearer\[dq],\[dq]refresh_token\[dq]:\[dq]yourrefreshtoken\[dq],\[dq]expiry\[dq]:\[dq]2018-08-26T22:39:52.486512262+08:00\[dq]} 50941 drive_id = b!Eqwertyuiopasdfghjklzxcvbnm-7mnbvcxzlkjhgfdsapoiuytrewqk 50942 drive_type = business -------------------- 50943 .IP "c)" 3 50944 Yes this is OK 50945 .IP "d)" 3 50946 Edit this remote 50947 .IP "e)" 3 50948 Delete this remote y/e/d> y 50949 .IP 50950 .nf 50951 \f[C] 50952 See the [remote setup docs](https://rclone.org/remote_setup/) for how to set it up on a 50953 machine with no Internet browser available. 50954 50955 Note that rclone runs a webserver on your local machine to collect the 50956 token as returned from Microsoft. This only runs from the moment it 50957 opens your browser to the moment you get back the verification 50958 code. This is on \[ga]http://127.0.0.1:53682/\[ga] and this it may require 50959 you to unblock it temporarily if you are running a host firewall. 50960 50961 Once configured you can then use \[ga]rclone\[ga] like this, 50962 50963 List directories in top level of your OneDrive 50964 50965 rclone lsd remote: 50966 50967 List all the files in your OneDrive 50968 50969 rclone ls remote: 50970 50971 To copy a local directory to an OneDrive directory called backup 50972 50973 rclone copy /home/source remote:backup 50974 50975 ### Getting your own Client ID and Key 50976 50977 rclone uses a default Client ID when talking to OneDrive, unless a custom \[ga]client_id\[ga] is specified in the config. 50978 The default Client ID and Key are shared by all rclone users when performing requests. 50979 50980 You may choose to create and use your own Client ID, in case the default one does not work well for you. 50981 For example, you might see throttling. 50982 50983 #### Creating Client ID for OneDrive Personal 50984 50985 To create your own Client ID, please follow these steps: 50986 50987 1. Open https://portal.azure.com/#blade/Microsoft_AAD_RegisteredApps/ApplicationsListBlade and then click \[ga]New registration\[ga]. 50988 2. Enter a name for your app, choose account type \[ga]Accounts in any organizational directory (Any Azure AD directory - Multitenant) and personal Microsoft accounts (e.g. Skype, Xbox)\[ga], select \[ga]Web\[ga] in \[ga]Redirect URI\[ga], then type (do not copy and paste) \[ga]http://localhost:53682/\[ga] and click Register. Copy and keep the \[ga]Application (client) ID\[ga] under the app name for later use. 50989 3. Under \[ga]manage\[ga] select \[ga]Certificates & secrets\[ga], click \[ga]New client secret\[ga]. Enter a description (can be anything) and set \[ga]Expires\[ga] to 24 months. Copy and keep that secret _Value_ for later use (you _won\[aq]t_ be able to see this value afterwards). 50990 4. Under \[ga]manage\[ga] select \[ga]API permissions\[ga], click \[ga]Add a permission\[ga] and select \[ga]Microsoft Graph\[ga] then select \[ga]delegated permissions\[ga]. 50991 5. Search and select the following permissions: \[ga]Files.Read\[ga], \[ga]Files.ReadWrite\[ga], \[ga]Files.Read.All\[ga], \[ga]Files.ReadWrite.All\[ga], \[ga]offline_access\[ga], \[ga]User.Read\[ga] and \[ga]Sites.Read.All\[ga] (if custom access scopes are configured, select the permissions accordingly). Once selected click \[ga]Add permissions\[ga] at the bottom. 50992 50993 Now the application is complete. Run \[ga]rclone config\[ga] to create or edit a OneDrive remote. 50994 Supply the app ID and password as Client ID and Secret, respectively. rclone will walk you through the remaining steps. 50995 50996 The access_scopes option allows you to configure the permissions requested by rclone. 50997 See [Microsoft Docs](https://docs.microsoft.com/en-us/graph/permissions-reference#files-permissions) for more information about the different scopes. 50998 50999 The \[ga]Sites.Read.All\[ga] permission is required if you need to [search SharePoint sites when configuring the remote](https://github.com/artpar/artpar/pull/5883). However, if that permission is not assigned, you need to exclude \[ga]Sites.Read.All\[ga] from your access scopes or set \[ga]disable_site_permission\[ga] option to true in the advanced options. 51000 51001 #### Creating Client ID for OneDrive Business 51002 51003 The steps for OneDrive Personal may or may not work for OneDrive Business, depending on the security settings of the organization. 51004 A common error is that the publisher of the App is not verified. 51005 51006 You may try to [verify you account](https://docs.microsoft.com/en-us/azure/active-directory/develop/publisher-verification-overview), or try to limit the App to your organization only, as shown below. 51007 51008 1. Make sure to create the App with your business account. 51009 2. Follow the steps above to create an App. However, we need a different account type here: \[ga]Accounts in this organizational directory only (*** - Single tenant)\[ga]. Note that you can also change the account type after creating the App. 51010 3. Find the [tenant ID](https://docs.microsoft.com/en-us/azure/active-directory/fundamentals/active-directory-how-to-find-tenant) of your organization. 51011 4. In the rclone config, set \[ga]auth_url\[ga] to \[ga]https://login.microsoftonline.com/YOUR_TENANT_ID/oauth2/v2.0/authorize\[ga]. 51012 5. In the rclone config, set \[ga]token_url\[ga] to \[ga]https://login.microsoftonline.com/YOUR_TENANT_ID/oauth2/v2.0/token\[ga]. 51013 51014 Note: If you have a special region, you may need a different host in step 4 and 5. Here are [some hints](https://github.com/artpar/artpar/blob/bc23bf11db1c78c6ebbf8ea538fbebf7058b4176/backend/onedrive/onedrive.go#L86). 51015 51016 51017 ### Modification times and hashes 51018 51019 OneDrive allows modification times to be set on objects accurate to 1 51020 second. These will be used to detect whether objects need syncing or 51021 not. 51022 51023 OneDrive Personal, OneDrive for Business and Sharepoint Server support 51024 [QuickXorHash](https://docs.microsoft.com/en-us/onedrive/developer/code-snippets/quickxorhash). 51025 51026 Before rclone 1.62 the default hash for Onedrive Personal was \[ga]SHA1\[ga]. 51027 For rclone 1.62 and above the default for all Onedrive backends is 51028 \[ga]QuickXorHash\[ga]. 51029 51030 Starting from July 2023 \[ga]SHA1\[ga] support is being phased out in Onedrive 51031 Personal in favour of \[ga]QuickXorHash\[ga]. If necessary the 51032 \[ga]--onedrive-hash-type\[ga] flag (or \[ga]hash_type\[ga] config option) can be used 51033 to select \[ga]SHA1\[ga] during the transition period if this is important 51034 your workflow. 51035 51036 For all types of OneDrive you can use the \[ga]--checksum\[ga] flag. 51037 51038 ### --fast-list 51039 51040 This remote supports \[ga]--fast-list\[ga] which allows you to use fewer 51041 transactions in exchange for more memory. See the [rclone 51042 docs](https://rclone.org/docs/#fast-list) for more details. 51043 51044 This must be enabled with the \[ga]--onedrive-delta\[ga] flag (or \[ga]delta = 51045 true\[ga] in the config file) as it can cause performance degradation. 51046 51047 It does this by using the delta listing facilities of OneDrive which 51048 returns all the files in the remote very efficiently. This is much 51049 more efficient than listing directories recursively and is Microsoft\[aq]s 51050 recommended way of reading all the file information from a drive. 51051 51052 This can be useful with \[ga]rclone mount\[ga] and [rclone rc vfs/refresh 51053 recursive=true](https://rclone.org/rc/#vfs-refresh)) to very quickly fill the mount with 51054 information about all the files. 51055 51056 The API used for the recursive listing (\[ga]ListR\[ga]) only supports listing 51057 from the root of the drive. This will become increasingly inefficient 51058 the further away you get from the root as rclone will have to discard 51059 files outside of the directory you are using. 51060 51061 Some commands (like \[ga]rclone lsf -R\[ga]) will use \[ga]ListR\[ga] by default - you 51062 can turn this off with \[ga]--disable ListR\[ga] if you need to. 51063 51064 ### Restricted filename characters 51065 51066 In addition to the [default restricted characters set](https://rclone.org/overview/#restricted-characters) 51067 the following characters are also replaced: 51068 51069 | Character | Value | Replacement | 51070 | --------- |:-----:|:-----------:| 51071 | \[dq] | 0x22 | \[uFF02] | 51072 | * | 0x2A | \[uFF0A] | 51073 | : | 0x3A | \[uFF1A] | 51074 | < | 0x3C | \[uFF1C] | 51075 | > | 0x3E | \[uFF1E] | 51076 | ? | 0x3F | \[uFF1F] | 51077 | \[rs] | 0x5C | \[uFF3C] | 51078 | \[rs]| | 0x7C | \[uFF5C] | 51079 51080 File names can also not end with the following characters. 51081 These only get replaced if they are the last character in the name: 51082 51083 | Character | Value | Replacement | 51084 | --------- |:-----:|:-----------:| 51085 | SP | 0x20 | \[u2420] | 51086 | . | 0x2E | \[uFF0E] | 51087 51088 File names can also not begin with the following characters. 51089 These only get replaced if they are the first character in the name: 51090 51091 | Character | Value | Replacement | 51092 | --------- |:-----:|:-----------:| 51093 | SP | 0x20 | \[u2420] | 51094 | \[ti] | 0x7E | \[uFF5E] | 51095 51096 Invalid UTF-8 bytes will also be [replaced](https://rclone.org/overview/#invalid-utf8), 51097 as they can\[aq]t be used in JSON strings. 51098 51099 ### Deleting files 51100 51101 Any files you delete with rclone will end up in the trash. Microsoft 51102 doesn\[aq]t provide an API to permanently delete files, nor to empty the 51103 trash, so you will have to do that with one of Microsoft\[aq]s apps or via 51104 the OneDrive website. 51105 51106 51107 ### Standard options 51108 51109 Here are the Standard options specific to onedrive (Microsoft OneDrive). 51110 51111 #### --onedrive-client-id 51112 51113 OAuth Client Id. 51114 51115 Leave blank normally. 51116 51117 Properties: 51118 51119 - Config: client_id 51120 - Env Var: RCLONE_ONEDRIVE_CLIENT_ID 51121 - Type: string 51122 - Required: false 51123 51124 #### --onedrive-client-secret 51125 51126 OAuth Client Secret. 51127 51128 Leave blank normally. 51129 51130 Properties: 51131 51132 - Config: client_secret 51133 - Env Var: RCLONE_ONEDRIVE_CLIENT_SECRET 51134 - Type: string 51135 - Required: false 51136 51137 #### --onedrive-region 51138 51139 Choose national cloud region for OneDrive. 51140 51141 Properties: 51142 51143 - Config: region 51144 - Env Var: RCLONE_ONEDRIVE_REGION 51145 - Type: string 51146 - Default: \[dq]global\[dq] 51147 - Examples: 51148 - \[dq]global\[dq] 51149 - Microsoft Cloud Global 51150 - \[dq]us\[dq] 51151 - Microsoft Cloud for US Government 51152 - \[dq]de\[dq] 51153 - Microsoft Cloud Germany 51154 - \[dq]cn\[dq] 51155 - Azure and Office 365 operated by Vnet Group in China 51156 51157 ### Advanced options 51158 51159 Here are the Advanced options specific to onedrive (Microsoft OneDrive). 51160 51161 #### --onedrive-token 51162 51163 OAuth Access Token as a JSON blob. 51164 51165 Properties: 51166 51167 - Config: token 51168 - Env Var: RCLONE_ONEDRIVE_TOKEN 51169 - Type: string 51170 - Required: false 51171 51172 #### --onedrive-auth-url 51173 51174 Auth server URL. 51175 51176 Leave blank to use the provider defaults. 51177 51178 Properties: 51179 51180 - Config: auth_url 51181 - Env Var: RCLONE_ONEDRIVE_AUTH_URL 51182 - Type: string 51183 - Required: false 51184 51185 #### --onedrive-token-url 51186 51187 Token server url. 51188 51189 Leave blank to use the provider defaults. 51190 51191 Properties: 51192 51193 - Config: token_url 51194 - Env Var: RCLONE_ONEDRIVE_TOKEN_URL 51195 - Type: string 51196 - Required: false 51197 51198 #### --onedrive-chunk-size 51199 51200 Chunk size to upload files with - must be multiple of 320k (327,680 bytes). 51201 51202 Above this size files will be chunked - must be multiple of 320k (327,680 bytes) and 51203 should not exceed 250M (262,144,000 bytes) else you may encounter \[rs]\[dq]Microsoft.SharePoint.Client.InvalidClientQueryException: The request message is too big.\[rs]\[dq] 51204 Note that the chunks will be buffered into memory. 51205 51206 Properties: 51207 51208 - Config: chunk_size 51209 - Env Var: RCLONE_ONEDRIVE_CHUNK_SIZE 51210 - Type: SizeSuffix 51211 - Default: 10Mi 51212 51213 #### --onedrive-drive-id 51214 51215 The ID of the drive to use. 51216 51217 Properties: 51218 51219 - Config: drive_id 51220 - Env Var: RCLONE_ONEDRIVE_DRIVE_ID 51221 - Type: string 51222 - Required: false 51223 51224 #### --onedrive-drive-type 51225 51226 The type of the drive (personal | business | documentLibrary). 51227 51228 Properties: 51229 51230 - Config: drive_type 51231 - Env Var: RCLONE_ONEDRIVE_DRIVE_TYPE 51232 - Type: string 51233 - Required: false 51234 51235 #### --onedrive-root-folder-id 51236 51237 ID of the root folder. 51238 51239 This isn\[aq]t normally needed, but in special circumstances you might 51240 know the folder ID that you wish to access but not be able to get 51241 there through a path traversal. 51242 51243 51244 Properties: 51245 51246 - Config: root_folder_id 51247 - Env Var: RCLONE_ONEDRIVE_ROOT_FOLDER_ID 51248 - Type: string 51249 - Required: false 51250 51251 #### --onedrive-access-scopes 51252 51253 Set scopes to be requested by rclone. 51254 51255 Choose or manually enter a custom space separated list with all scopes, that rclone should request. 51256 51257 51258 Properties: 51259 51260 - Config: access_scopes 51261 - Env Var: RCLONE_ONEDRIVE_ACCESS_SCOPES 51262 - Type: SpaceSepList 51263 - Default: Files.Read Files.ReadWrite Files.Read.All Files.ReadWrite.All Sites.Read.All offline_access 51264 - Examples: 51265 - \[dq]Files.Read Files.ReadWrite Files.Read.All Files.ReadWrite.All Sites.Read.All offline_access\[dq] 51266 - Read and write access to all resources 51267 - \[dq]Files.Read Files.Read.All Sites.Read.All offline_access\[dq] 51268 - Read only access to all resources 51269 - \[dq]Files.Read Files.ReadWrite Files.Read.All Files.ReadWrite.All offline_access\[dq] 51270 - Read and write access to all resources, without the ability to browse SharePoint sites. 51271 - Same as if disable_site_permission was set to true 51272 51273 #### --onedrive-disable-site-permission 51274 51275 Disable the request for Sites.Read.All permission. 51276 51277 If set to true, you will no longer be able to search for a SharePoint site when 51278 configuring drive ID, because rclone will not request Sites.Read.All permission. 51279 Set it to true if your organization didn\[aq]t assign Sites.Read.All permission to the 51280 application, and your organization disallows users to consent app permission 51281 request on their own. 51282 51283 Properties: 51284 51285 - Config: disable_site_permission 51286 - Env Var: RCLONE_ONEDRIVE_DISABLE_SITE_PERMISSION 51287 - Type: bool 51288 - Default: false 51289 51290 #### --onedrive-expose-onenote-files 51291 51292 Set to make OneNote files show up in directory listings. 51293 51294 By default, rclone will hide OneNote files in directory listings because 51295 operations like \[dq]Open\[dq] and \[dq]Update\[dq] won\[aq]t work on them. But this 51296 behaviour may also prevent you from deleting them. If you want to 51297 delete OneNote files or otherwise want them to show up in directory 51298 listing, set this option. 51299 51300 Properties: 51301 51302 - Config: expose_onenote_files 51303 - Env Var: RCLONE_ONEDRIVE_EXPOSE_ONENOTE_FILES 51304 - Type: bool 51305 - Default: false 51306 51307 #### --onedrive-server-side-across-configs 51308 51309 Deprecated: use --server-side-across-configs instead. 51310 51311 Allow server-side operations (e.g. copy) to work across different onedrive configs. 51312 51313 This will only work if you are copying between two OneDrive *Personal* drives AND 51314 the files to copy are already shared between them. In other cases, rclone will 51315 fall back to normal copy (which will be slightly slower). 51316 51317 Properties: 51318 51319 - Config: server_side_across_configs 51320 - Env Var: RCLONE_ONEDRIVE_SERVER_SIDE_ACROSS_CONFIGS 51321 - Type: bool 51322 - Default: false 51323 51324 #### --onedrive-list-chunk 51325 51326 Size of listing chunk. 51327 51328 Properties: 51329 51330 - Config: list_chunk 51331 - Env Var: RCLONE_ONEDRIVE_LIST_CHUNK 51332 - Type: int 51333 - Default: 1000 51334 51335 #### --onedrive-no-versions 51336 51337 Remove all versions on modifying operations. 51338 51339 Onedrive for business creates versions when rclone uploads new files 51340 overwriting an existing one and when it sets the modification time. 51341 51342 These versions take up space out of the quota. 51343 51344 This flag checks for versions after file upload and setting 51345 modification time and removes all but the last version. 51346 51347 **NB** Onedrive personal can\[aq]t currently delete versions so don\[aq]t use 51348 this flag there. 51349 51350 51351 Properties: 51352 51353 - Config: no_versions 51354 - Env Var: RCLONE_ONEDRIVE_NO_VERSIONS 51355 - Type: bool 51356 - Default: false 51357 51358 #### --onedrive-link-scope 51359 51360 Set the scope of the links created by the link command. 51361 51362 Properties: 51363 51364 - Config: link_scope 51365 - Env Var: RCLONE_ONEDRIVE_LINK_SCOPE 51366 - Type: string 51367 - Default: \[dq]anonymous\[dq] 51368 - Examples: 51369 - \[dq]anonymous\[dq] 51370 - Anyone with the link has access, without needing to sign in. 51371 - This may include people outside of your organization. 51372 - Anonymous link support may be disabled by an administrator. 51373 - \[dq]organization\[dq] 51374 - Anyone signed into your organization (tenant) can use the link to get access. 51375 - Only available in OneDrive for Business and SharePoint. 51376 51377 #### --onedrive-link-type 51378 51379 Set the type of the links created by the link command. 51380 51381 Properties: 51382 51383 - Config: link_type 51384 - Env Var: RCLONE_ONEDRIVE_LINK_TYPE 51385 - Type: string 51386 - Default: \[dq]view\[dq] 51387 - Examples: 51388 - \[dq]view\[dq] 51389 - Creates a read-only link to the item. 51390 - \[dq]edit\[dq] 51391 - Creates a read-write link to the item. 51392 - \[dq]embed\[dq] 51393 - Creates an embeddable link to the item. 51394 51395 #### --onedrive-link-password 51396 51397 Set the password for links created by the link command. 51398 51399 At the time of writing this only works with OneDrive personal paid accounts. 51400 51401 51402 Properties: 51403 51404 - Config: link_password 51405 - Env Var: RCLONE_ONEDRIVE_LINK_PASSWORD 51406 - Type: string 51407 - Required: false 51408 51409 #### --onedrive-hash-type 51410 51411 Specify the hash in use for the backend. 51412 51413 This specifies the hash type in use. If set to \[dq]auto\[dq] it will use the 51414 default hash which is QuickXorHash. 51415 51416 Before rclone 1.62 an SHA1 hash was used by default for Onedrive 51417 Personal. For 1.62 and later the default is to use a QuickXorHash for 51418 all onedrive types. If an SHA1 hash is desired then set this option 51419 accordingly. 51420 51421 From July 2023 QuickXorHash will be the only available hash for 51422 both OneDrive for Business and OneDriver Personal. 51423 51424 This can be set to \[dq]none\[dq] to not use any hashes. 51425 51426 If the hash requested does not exist on the object, it will be 51427 returned as an empty string which is treated as a missing hash by 51428 rclone. 51429 51430 51431 Properties: 51432 51433 - Config: hash_type 51434 - Env Var: RCLONE_ONEDRIVE_HASH_TYPE 51435 - Type: string 51436 - Default: \[dq]auto\[dq] 51437 - Examples: 51438 - \[dq]auto\[dq] 51439 - Rclone chooses the best hash 51440 - \[dq]quickxor\[dq] 51441 - QuickXor 51442 - \[dq]sha1\[dq] 51443 - SHA1 51444 - \[dq]sha256\[dq] 51445 - SHA256 51446 - \[dq]crc32\[dq] 51447 - CRC32 51448 - \[dq]none\[dq] 51449 - None - don\[aq]t use any hashes 51450 51451 #### --onedrive-av-override 51452 51453 Allows download of files the server thinks has a virus. 51454 51455 The onedrive/sharepoint server may check files uploaded with an Anti 51456 Virus checker. If it detects any potential viruses or malware it will 51457 block download of the file. 51458 51459 In this case you will see a message like this 51460 51461 server reports this file is infected with a virus - use --onedrive-av-override to download anyway: Infected (name of virus): 403 Forbidden: 51462 51463 If you are 100% sure you want to download this file anyway then use 51464 the --onedrive-av-override flag, or av_override = true in the config 51465 file. 51466 51467 51468 Properties: 51469 51470 - Config: av_override 51471 - Env Var: RCLONE_ONEDRIVE_AV_OVERRIDE 51472 - Type: bool 51473 - Default: false 51474 51475 #### --onedrive-delta 51476 51477 If set rclone will use delta listing to implement recursive listings. 51478 51479 If this flag is set the onedrive backend will advertise \[ga]ListR\[ga] 51480 support for recursive listings. 51481 51482 Setting this flag speeds up these things greatly: 51483 51484 rclone lsf -R onedrive: 51485 rclone size onedrive: 51486 rclone rc vfs/refresh recursive=true 51487 51488 **However** the delta listing API **only** works at the root of the 51489 drive. If you use it not at the root then it recurses from the root 51490 and discards all the data that is not under the directory you asked 51491 for. So it will be correct but may not be very efficient. 51492 51493 This is why this flag is not set as the default. 51494 51495 As a rule of thumb if nearly all of your data is under rclone\[aq]s root 51496 directory (the \[ga]root/directory\[ga] in \[ga]onedrive:root/directory\[ga]) then 51497 using this flag will be be a big performance win. If your data is 51498 mostly not under the root then using this flag will be a big 51499 performance loss. 51500 51501 It is recommended if you are mounting your onedrive at the root 51502 (or near the root when using crypt) and using rclone \[ga]rc vfs/refresh\[ga]. 51503 51504 51505 Properties: 51506 51507 - Config: delta 51508 - Env Var: RCLONE_ONEDRIVE_DELTA 51509 - Type: bool 51510 - Default: false 51511 51512 #### --onedrive-metadata-permissions 51513 51514 Control whether permissions should be read or written in metadata. 51515 51516 Reading permissions metadata from files can be done quickly, but it 51517 isn\[aq]t always desirable to set the permissions from the metadata. 51518 51519 51520 Properties: 51521 51522 - Config: metadata_permissions 51523 - Env Var: RCLONE_ONEDRIVE_METADATA_PERMISSIONS 51524 - Type: Bits 51525 - Default: off 51526 - Examples: 51527 - \[dq]off\[dq] 51528 - Do not read or write the value 51529 - \[dq]read\[dq] 51530 - Read the value only 51531 - \[dq]write\[dq] 51532 - Write the value only 51533 - \[dq]read,write\[dq] 51534 - Read and Write the value. 51535 51536 #### --onedrive-encoding 51537 51538 The encoding for the backend. 51539 51540 See the [encoding section in the overview](https://rclone.org/overview/#encoding) for more info. 51541 51542 Properties: 51543 51544 - Config: encoding 51545 - Env Var: RCLONE_ONEDRIVE_ENCODING 51546 - Type: Encoding 51547 - Default: Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Del,Ctl,LeftSpace,LeftTilde,RightSpace,RightPeriod,InvalidUtf8,Dot 51548 51549 #### --onedrive-description 51550 51551 Description of the remote 51552 51553 Properties: 51554 51555 - Config: description 51556 - Env Var: RCLONE_ONEDRIVE_DESCRIPTION 51557 - Type: string 51558 - Required: false 51559 51560 ### Metadata 51561 51562 OneDrive supports System Metadata (not User Metadata, as of this writing) for 51563 both files and directories. Much of the metadata is read-only, and there are some 51564 differences between OneDrive Personal and Business (see table below for 51565 details). 51566 51567 Permissions are also supported, if \[ga]--onedrive-metadata-permissions\[ga] is set. The 51568 accepted values for \[ga]--onedrive-metadata-permissions\[ga] are \[ga]read\[ga], \[ga]write\[ga], 51569 \[ga]read,write\[ga], and \[ga]off\[ga] (the default). \[ga]write\[ga] supports adding new permissions, 51570 updating the \[dq]role\[dq] of existing permissions, and removing permissions. Updating 51571 and removing require the Permission ID to be known, so it is recommended to use 51572 \[ga]read,write\[ga] instead of \[ga]write\[ga] if you wish to update/remove permissions. 51573 51574 Permissions are read/written in JSON format using the same schema as the 51575 [OneDrive API](https://learn.microsoft.com/en-us/onedrive/developer/rest-api/resources/permission?view=odsp-graph-online), 51576 which differs slightly between OneDrive Personal and Business. 51577 51578 Example for OneDrive Personal: 51579 \[ga]\[ga]\[ga]json 51580 [ 51581 { 51582 \[dq]id\[dq]: \[dq]1234567890ABC!123\[dq], 51583 \[dq]grantedTo\[dq]: { 51584 \[dq]user\[dq]: { 51585 \[dq]id\[dq]: \[dq]ryan\[at]contoso.com\[dq] 51586 }, 51587 \[dq]application\[dq]: {}, 51588 \[dq]device\[dq]: {} 51589 }, 51590 \[dq]invitation\[dq]: { 51591 \[dq]email\[dq]: \[dq]ryan\[at]contoso.com\[dq] 51592 }, 51593 \[dq]link\[dq]: { 51594 \[dq]webUrl\[dq]: \[dq]https://1drv.ms/t/s!1234567890ABC\[dq] 51595 }, 51596 \[dq]roles\[dq]: [ 51597 \[dq]read\[dq] 51598 ], 51599 \[dq]shareId\[dq]: \[dq]s!1234567890ABC\[dq] 51600 } 51601 ] 51602 \f[R] 51603 .fi 51604 .PP 51605 Example for OneDrive Business: 51606 .IP 51607 .nf 51608 \f[C] 51609 [ 51610 { 51611 \[dq]id\[dq]: \[dq]48d31887-5fad-4d73-a9f5-3c356e68a038\[dq], 51612 \[dq]grantedToIdentities\[dq]: [ 51613 { 51614 \[dq]user\[dq]: { 51615 \[dq]displayName\[dq]: \[dq]ryan\[at]contoso.com\[dq] 51616 }, 51617 \[dq]application\[dq]: {}, 51618 \[dq]device\[dq]: {} 51619 } 51620 ], 51621 \[dq]link\[dq]: { 51622 \[dq]type\[dq]: \[dq]view\[dq], 51623 \[dq]scope\[dq]: \[dq]users\[dq], 51624 \[dq]webUrl\[dq]: \[dq]https://contoso.sharepoint.com/:w:/t/design/a577ghg9hgh737613bmbjf839026561fmzhsr85ng9f3hjck2t5s\[dq] 51625 }, 51626 \[dq]roles\[dq]: [ 51627 \[dq]read\[dq] 51628 ], 51629 \[dq]shareId\[dq]: \[dq]u!LKj1lkdlals90j1nlkascl\[dq] 51630 }, 51631 { 51632 \[dq]id\[dq]: \[dq]5D33DD65C6932946\[dq], 51633 \[dq]grantedTo\[dq]: { 51634 \[dq]user\[dq]: { 51635 \[dq]displayName\[dq]: \[dq]John Doe\[dq], 51636 \[dq]id\[dq]: \[dq]efee1b77-fb3b-4f65-99d6-274c11914d12\[dq] 51637 }, 51638 \[dq]application\[dq]: {}, 51639 \[dq]device\[dq]: {} 51640 }, 51641 \[dq]roles\[dq]: [ 51642 \[dq]owner\[dq] 51643 ], 51644 \[dq]shareId\[dq]: \[dq]FWxc1lasfdbEAGM5fI7B67aB5ZMPDMmQ11U\[dq] 51645 } 51646 ] 51647 \f[R] 51648 .fi 51649 .PP 51650 To write permissions, pass in a \[dq]permissions\[dq] metadata key using 51651 this same format. 51652 The 51653 \f[C]--metadata-mapper\f[R] (https://rclone.org/docs/#metadata-mapper) 51654 tool can be very helpful for this. 51655 .PP 51656 When adding permissions, an email address can be provided in the 51657 \f[C]User.ID\f[R] or \f[C]DisplayName\f[R] properties of 51658 \f[C]grantedTo\f[R] or \f[C]grantedToIdentities\f[R]. 51659 Alternatively, an ObjectID can be provided in \f[C]User.ID\f[R]. 51660 At least one valid recipient must be provided in order to add a 51661 permission for a user. 51662 Creating a Public Link is also supported, if \f[C]Link.Scope\f[R] is set 51663 to \f[C]\[dq]anonymous\[dq]\f[R]. 51664 .PP 51665 Example request to add a \[dq]read\[dq] permission: 51666 .IP 51667 .nf 51668 \f[C] 51669 [ 51670 { 51671 \[dq]id\[dq]: \[dq]\[dq], 51672 \[dq]grantedTo\[dq]: { 51673 \[dq]user\[dq]: {}, 51674 \[dq]application\[dq]: {}, 51675 \[dq]device\[dq]: {} 51676 }, 51677 \[dq]grantedToIdentities\[dq]: [ 51678 { 51679 \[dq]user\[dq]: { 51680 \[dq]id\[dq]: \[dq]ryan\[at]contoso.com\[dq] 51681 }, 51682 \[dq]application\[dq]: {}, 51683 \[dq]device\[dq]: {} 51684 } 51685 ], 51686 \[dq]roles\[dq]: [ 51687 \[dq]read\[dq] 51688 ] 51689 } 51690 ] 51691 \f[R] 51692 .fi 51693 .PP 51694 Note that adding a permission can fail if a conflicting permission 51695 already exists for the file/folder. 51696 .PP 51697 To update an existing permission, include both the Permission ID and the 51698 new \f[C]roles\f[R] to be assigned. 51699 \f[C]roles\f[R] is the only property that can be changed. 51700 .PP 51701 To remove permissions, pass in a blob containing only the permissions 51702 you wish to keep (which can be empty, to remove all.) 51703 .PP 51704 Note that both reading and writing permissions requires extra API calls, 51705 so if you don\[aq]t need to read or write permissions it is recommended 51706 to omit \f[C]--onedrive-metadata-permissions\f[R]. 51707 .PP 51708 Metadata and permissions are supported for Folders (directories) as well 51709 as Files. 51710 Note that setting the \f[C]mtime\f[R] or \f[C]btime\f[R] on a Folder 51711 requires one extra API call on OneDrive Business only. 51712 .PP 51713 OneDrive does not currently support User Metadata. 51714 When writing metadata, only writeable system properties will be written 51715 -- any read-only or unrecognized keys passed in will be ignored. 51716 .PP 51717 TIP: to see the metadata and permissions for any file or folder, run: 51718 .IP 51719 .nf 51720 \f[C] 51721 rclone lsjson remote:path --stat -M --onedrive-metadata-permissions read 51722 \f[R] 51723 .fi 51724 .PP 51725 Here are the possible system metadata items for the onedrive backend. 51726 .PP 51727 .TS 51728 tab(@); 51729 lw(11.1n) lw(11.1n) lw(11.1n) lw(16.6n) lw(20.3n). 51730 T{ 51731 Name 51732 T}@T{ 51733 Help 51734 T}@T{ 51735 Type 51736 T}@T{ 51737 Example 51738 T}@T{ 51739 Read Only 51740 T} 51741 _ 51742 T{ 51743 btime 51744 T}@T{ 51745 Time of file birth (creation) with S accuracy (mS for OneDrive 51746 Personal). 51747 T}@T{ 51748 RFC 3339 51749 T}@T{ 51750 2006-01-02T15:04:05Z 51751 T}@T{ 51752 N 51753 T} 51754 T{ 51755 content-type 51756 T}@T{ 51757 The MIME type of the file. 51758 T}@T{ 51759 string 51760 T}@T{ 51761 text/plain 51762 T}@T{ 51763 \f[B]Y\f[R] 51764 T} 51765 T{ 51766 created-by-display-name 51767 T}@T{ 51768 Display name of the user that created the item. 51769 T}@T{ 51770 string 51771 T}@T{ 51772 John Doe 51773 T}@T{ 51774 \f[B]Y\f[R] 51775 T} 51776 T{ 51777 created-by-id 51778 T}@T{ 51779 ID of the user that created the item. 51780 T}@T{ 51781 string 51782 T}@T{ 51783 48d31887-5fad-4d73-a9f5-3c356e68a038 51784 T}@T{ 51785 \f[B]Y\f[R] 51786 T} 51787 T{ 51788 description 51789 T}@T{ 51790 A short description of the file. 51791 Max 1024 characters. 51792 Only supported for OneDrive Personal. 51793 T}@T{ 51794 string 51795 T}@T{ 51796 Contract for signing 51797 T}@T{ 51798 N 51799 T} 51800 T{ 51801 id 51802 T}@T{ 51803 The unique identifier of the item within OneDrive. 51804 T}@T{ 51805 string 51806 T}@T{ 51807 01BYE5RZ6QN3ZWBTUFOFD3GSPGOHDJD36K 51808 T}@T{ 51809 \f[B]Y\f[R] 51810 T} 51811 T{ 51812 last-modified-by-display-name 51813 T}@T{ 51814 Display name of the user that last modified the item. 51815 T}@T{ 51816 string 51817 T}@T{ 51818 John Doe 51819 T}@T{ 51820 \f[B]Y\f[R] 51821 T} 51822 T{ 51823 last-modified-by-id 51824 T}@T{ 51825 ID of the user that last modified the item. 51826 T}@T{ 51827 string 51828 T}@T{ 51829 48d31887-5fad-4d73-a9f5-3c356e68a038 51830 T}@T{ 51831 \f[B]Y\f[R] 51832 T} 51833 T{ 51834 malware-detected 51835 T}@T{ 51836 Whether OneDrive has detected that the item contains malware. 51837 T}@T{ 51838 boolean 51839 T}@T{ 51840 true 51841 T}@T{ 51842 \f[B]Y\f[R] 51843 T} 51844 T{ 51845 mtime 51846 T}@T{ 51847 Time of last modification with S accuracy (mS for OneDrive Personal). 51848 T}@T{ 51849 RFC 3339 51850 T}@T{ 51851 2006-01-02T15:04:05Z 51852 T}@T{ 51853 N 51854 T} 51855 T{ 51856 package-type 51857 T}@T{ 51858 If present, indicates that this item is a package instead of a folder or 51859 file. 51860 Packages are treated like files in some contexts and folders in others. 51861 T}@T{ 51862 string 51863 T}@T{ 51864 oneNote 51865 T}@T{ 51866 \f[B]Y\f[R] 51867 T} 51868 T{ 51869 permissions 51870 T}@T{ 51871 Permissions in a JSON dump of OneDrive format. 51872 Enable with --onedrive-metadata-permissions. 51873 Properties: id, grantedTo, grantedToIdentities, invitation, 51874 inheritedFrom, link, roles, shareId 51875 T}@T{ 51876 JSON 51877 T}@T{ 51878 {} 51879 T}@T{ 51880 N 51881 T} 51882 T{ 51883 shared-by-id 51884 T}@T{ 51885 ID of the user that shared the item (if shared). 51886 T}@T{ 51887 string 51888 T}@T{ 51889 48d31887-5fad-4d73-a9f5-3c356e68a038 51890 T}@T{ 51891 \f[B]Y\f[R] 51892 T} 51893 T{ 51894 shared-owner-id 51895 T}@T{ 51896 ID of the owner of the shared item (if shared). 51897 T}@T{ 51898 string 51899 T}@T{ 51900 48d31887-5fad-4d73-a9f5-3c356e68a038 51901 T}@T{ 51902 \f[B]Y\f[R] 51903 T} 51904 T{ 51905 shared-scope 51906 T}@T{ 51907 If shared, indicates the scope of how the item is shared: anonymous, 51908 organization, or users. 51909 T}@T{ 51910 string 51911 T}@T{ 51912 users 51913 T}@T{ 51914 \f[B]Y\f[R] 51915 T} 51916 T{ 51917 shared-time 51918 T}@T{ 51919 Time when the item was shared, with S accuracy (mS for OneDrive 51920 Personal). 51921 T}@T{ 51922 RFC 3339 51923 T}@T{ 51924 2006-01-02T15:04:05Z 51925 T}@T{ 51926 \f[B]Y\f[R] 51927 T} 51928 T{ 51929 utime 51930 T}@T{ 51931 Time of upload with S accuracy (mS for OneDrive Personal). 51932 T}@T{ 51933 RFC 3339 51934 T}@T{ 51935 2006-01-02T15:04:05Z 51936 T}@T{ 51937 \f[B]Y\f[R] 51938 T} 51939 .TE 51940 .PP 51941 See the metadata (https://rclone.org/docs/#metadata) docs for more info. 51942 .SS Limitations 51943 .PP 51944 If you don\[aq]t use rclone for 90 days the refresh token will expire. 51945 This will result in authorization problems. 51946 This is easy to fix by running the 51947 \f[C]rclone config reconnect remote:\f[R] command to get a new token and 51948 refresh token. 51949 .SS Naming 51950 .PP 51951 Note that OneDrive is case insensitive so you can\[aq]t have a file 51952 called \[dq]Hello.doc\[dq] and one called \[dq]hello.doc\[dq]. 51953 .PP 51954 There are quite a few characters that can\[aq]t be in OneDrive file 51955 names. 51956 These can\[aq]t occur on Windows platforms, but on non-Windows platforms 51957 they are common. 51958 Rclone will map these names to and from an identical looking unicode 51959 equivalent. 51960 For example if a file has a \f[C]?\f[R] in it will be mapped to 51961 \f[C]\[uFF1F]\f[R] instead. 51962 .SS File sizes 51963 .PP 51964 The largest allowed file size is 250 GiB for both OneDrive Personal and 51965 OneDrive for Business (Updated 13 Jan 51966 2021) (https://support.microsoft.com/en-us/office/invalid-file-names-and-file-types-in-onedrive-and-sharepoint-64883a5d-228e-48f5-b3d2-eb39e07630fa?ui=en-us&rs=en-us&ad=us#individualfilesize). 51967 .SS Path length 51968 .PP 51969 The entire path, including the file name, must contain fewer than 400 51970 characters for OneDrive, OneDrive for Business and SharePoint Online. 51971 If you are encrypting file and folder names with rclone, you may want to 51972 pay attention to this limitation because the encrypted names are 51973 typically longer than the original ones. 51974 .SS Number of files 51975 .PP 51976 OneDrive seems to be OK with at least 50,000 files in a folder, but at 51977 100,000 rclone will get errors listing the directory like 51978 \f[C]couldn\[cq]t list files: UnknownError:\f[R]. 51979 See #2707 (https://github.com/artpar/artpar/issues/2707) for more info. 51980 .PP 51981 An official document about the limitations for different types of 51982 OneDrive can be found 51983 here (https://support.office.com/en-us/article/invalid-file-names-and-file-types-in-onedrive-onedrive-for-business-and-sharepoint-64883a5d-228e-48f5-b3d2-eb39e07630fa). 51984 .SS Versions 51985 .PP 51986 Every change in a file OneDrive causes the service to create a new 51987 version of the file. 51988 This counts against a users quota. 51989 For example changing the modification time of a file creates a second 51990 version, so the file apparently uses twice the space. 51991 .PP 51992 For example the \f[C]copy\f[R] command is affected by this as rclone 51993 copies the file and then afterwards sets the modification time to match 51994 the source file which uses another version. 51995 .PP 51996 You can use the \f[C]rclone cleanup\f[R] command (see below) to remove 51997 all old versions. 51998 .PP 51999 Or you can set the \f[C]no_versions\f[R] parameter to \f[C]true\f[R] and 52000 rclone will remove versions after operations which create new versions. 52001 This takes extra transactions so only enable it if you need it. 52002 .PP 52003 \f[B]Note\f[R] At the time of writing Onedrive Personal creates versions 52004 (but not for setting the modification time) but the API for removing 52005 them returns \[dq]API not found\[dq] so cleanup and 52006 \f[C]no_versions\f[R] should not be used on Onedrive Personal. 52007 .SS Disabling versioning 52008 .PP 52009 Starting October 2018, users will no longer be able to disable 52010 versioning by default. 52011 This is because Microsoft has brought an 52012 update (https://techcommunity.microsoft.com/t5/Microsoft-OneDrive-Blog/New-Updates-to-OneDrive-and-SharePoint-Team-Site-Versioning/ba-p/204390) 52013 to the mechanism. 52014 To change this new default setting, a PowerShell command is required to 52015 be run by a SharePoint admin. 52016 If you are an admin, you can run these commands in PowerShell to change 52017 that setting: 52018 .IP "1." 3 52019 \f[C]Install-Module -Name Microsoft.Online.SharePoint.PowerShell\f[R] 52020 (in case you haven\[aq]t installed this already) 52021 .IP "2." 3 52022 \f[C]Import-Module Microsoft.Online.SharePoint.PowerShell -DisableNameChecking\f[R] 52023 .IP "3." 3 52024 \f[C]Connect-SPOService -Url https://YOURSITE-admin.sharepoint.com -Credential YOU\[at]YOURSITE.COM\f[R] 52025 (replacing \f[C]YOURSITE\f[R], \f[C]YOU\f[R], \f[C]YOURSITE.COM\f[R] 52026 with the actual values; this will prompt for your credentials) 52027 .IP "4." 3 52028 \f[C]Set-SPOTenant -EnableMinimumVersionRequirement $False\f[R] 52029 .IP "5." 3 52030 \f[C]Disconnect-SPOService\f[R] (to disconnect from the server) 52031 .PP 52032 \f[I]Below are the steps for normal users to disable versioning. If you 52033 don\[aq]t see the \[dq]No Versioning\[dq] option, make sure the above 52034 requirements are met.\f[R] 52035 .PP 52036 User Weropol (https://github.com/Weropol) has found a method to disable 52037 versioning on OneDrive 52038 .IP "1." 3 52039 Open the settings menu by clicking on the gear symbol at the top of the 52040 OneDrive Business page. 52041 .IP "2." 3 52042 Click Site settings. 52043 .IP "3." 3 52044 Once on the Site settings page, navigate to Site Administration > Site 52045 libraries and lists. 52046 .IP "4." 3 52047 Click Customize \[dq]Documents\[dq]. 52048 .IP "5." 3 52049 Click General Settings > Versioning Settings. 52050 .IP "6." 3 52051 Under Document Version History select the option No versioning. 52052 Note: This will disable the creation of new file versions, but will not 52053 remove any previous versions. 52054 Your documents are safe. 52055 .IP "7." 3 52056 Apply the changes by clicking OK. 52057 .IP "8." 3 52058 Use rclone to upload or modify files. 52059 (I also use the --no-update-modtime flag) 52060 .IP "9." 3 52061 Restore the versioning settings after using rclone. 52062 (Optional) 52063 .SS Cleanup 52064 .PP 52065 OneDrive supports \f[C]rclone cleanup\f[R] which causes rclone to look 52066 through every file under the path supplied and delete all version but 52067 the current version. 52068 Because this involves traversing all the files, then querying each file 52069 for versions it can be quite slow. 52070 Rclone does \f[C]--checkers\f[R] tests in parallel. 52071 The command also supports \f[C]--interactive\f[R]/\f[C]i\f[R] or 52072 \f[C]--dry-run\f[R] which is a great way to see what it would do. 52073 .IP 52074 .nf 52075 \f[C] 52076 rclone cleanup --interactive remote:path/subdir # interactively remove all old version for path/subdir 52077 rclone cleanup remote:path/subdir # unconditionally remove all old version for path/subdir 52078 \f[R] 52079 .fi 52080 .PP 52081 \f[B]NB\f[R] Onedrive personal can\[aq]t currently delete versions 52082 .SS Troubleshooting 52083 .SS Excessive throttling or blocked on SharePoint 52084 .PP 52085 If you experience excessive throttling or is being blocked on SharePoint 52086 then it may help to set the user agent explicitly with a flag like this: 52087 \f[C]--user-agent \[dq]ISV|rclone.org|rclone/v1.55.1\[dq]\f[R] 52088 .PP 52089 The specific details can be found in the Microsoft document: Avoid 52090 getting throttled or blocked in SharePoint 52091 Online (https://docs.microsoft.com/en-us/sharepoint/dev/general-development/how-to-avoid-getting-throttled-or-blocked-in-sharepoint-online#how-to-decorate-your-http-traffic-to-avoid-throttling) 52092 .SS Unexpected file size/hash differences on Sharepoint 52093 .PP 52094 It is a 52095 known (https://github.com/OneDrive/onedrive-api-docs/issues/935#issuecomment-441741631) 52096 issue that Sharepoint (not OneDrive or OneDrive for Business) silently 52097 modifies uploaded files, mainly Office files (.docx, .xlsx, etc.), 52098 causing file size and hash checks to fail. 52099 There are also other situations that will cause OneDrive to report 52100 inconsistent file sizes. 52101 To use rclone with such affected files on Sharepoint, you may disable 52102 these checks with the following command line arguments: 52103 .IP 52104 .nf 52105 \f[C] 52106 --ignore-checksum --ignore-size 52107 \f[R] 52108 .fi 52109 .PP 52110 Alternatively, if you have write access to the OneDrive files, it may be 52111 possible to fix this problem for certain files, by attempting the steps 52112 below. 52113 Open the web interface for OneDrive (https://onedrive.live.com) and find 52114 the affected files (which will be in the error messages/log for rclone). 52115 Simply click on each of these files, causing OneDrive to open them on 52116 the web. 52117 This will cause each file to be converted in place to a format that is 52118 functionally equivalent but which will no longer trigger the size 52119 discrepancy. 52120 Once all problematic files are converted you will no longer need the 52121 ignore options above. 52122 .SS Replacing/deleting existing files on Sharepoint gets \[dq]item not found\[dq] 52123 .PP 52124 It is a 52125 known (https://github.com/OneDrive/onedrive-api-docs/issues/1068) issue 52126 that Sharepoint (not OneDrive or OneDrive for Business) may return 52127 \[dq]item not found\[dq] errors when users try to replace or delete 52128 uploaded files; this seems to mainly affect Office files (.docx, .xlsx, 52129 etc.) and web files (.html, .aspx, etc.). 52130 As a workaround, you may use the \f[C]--backup-dir <BACKUP_DIR>\f[R] 52131 command line argument so rclone moves the files to be replaced/deleted 52132 into a given backup directory (instead of directly replacing/deleting 52133 them). 52134 For example, to instruct rclone to move the files into the directory 52135 \f[C]rclone-backup-dir\f[R] on backend \f[C]mysharepoint\f[R], you may 52136 use: 52137 .IP 52138 .nf 52139 \f[C] 52140 --backup-dir mysharepoint:rclone-backup-dir 52141 \f[R] 52142 .fi 52143 .SS access_denied (AADSTS65005) 52144 .IP 52145 .nf 52146 \f[C] 52147 Error: access_denied 52148 Code: AADSTS65005 52149 Description: Using application \[aq]rclone\[aq] is currently not supported for your organization [YOUR_ORGANIZATION] because it is in an unmanaged state. An administrator needs to claim ownership of the company by DNS validation of [YOUR_ORGANIZATION] before the application rclone can be provisioned. 52150 \f[R] 52151 .fi 52152 .PP 52153 This means that rclone can\[aq]t use the OneDrive for Business API with 52154 your account. 52155 You can\[aq]t do much about it, maybe write an email to your admins. 52156 .PP 52157 However, there are other ways to interact with your OneDrive account. 52158 Have a look at the WebDAV backend: https://rclone.org/webdav/#sharepoint 52159 .SS invalid_grant (AADSTS50076) 52160 .IP 52161 .nf 52162 \f[C] 52163 Error: invalid_grant 52164 Code: AADSTS50076 52165 Description: Due to a configuration change made by your administrator, or because you moved to a new location, you must use multi-factor authentication to access \[aq]...\[aq]. 52166 \f[R] 52167 .fi 52168 .PP 52169 If you see the error above after enabling multi-factor authentication 52170 for your account, you can fix it by refreshing your OAuth refresh token. 52171 To do that, run \f[C]rclone config\f[R], and choose to edit your 52172 OneDrive backend. 52173 Then, you don\[aq]t need to actually make any changes until you reach 52174 this question: \f[C]Already have a token - refresh?\f[R]. 52175 For this question, answer \f[C]y\f[R] and go through the process to 52176 refresh your token, just like the first time the backend is configured. 52177 After this, rclone should work again for this backend. 52178 .SS Invalid request when making public links 52179 .PP 52180 On Sharepoint and OneDrive for Business, \f[C]rclone link\f[R] may 52181 return an \[dq]Invalid request\[dq] error. 52182 A possible cause is that the organisation admin didn\[aq]t allow public 52183 links to be made for the organisation/sharepoint library. 52184 To fix the permissions as an admin, take a look at the docs: 52185 1 (https://docs.microsoft.com/en-us/sharepoint/turn-external-sharing-on-or-off), 52186 2 (https://support.microsoft.com/en-us/office/set-up-and-manage-access-requests-94b26e0b-2822-49d4-929a-8455698654b3). 52187 .SS Can not access \f[C]Shared\f[R] with me files 52188 .PP 52189 Shared with me files is not supported by rclone 52190 currently (https://github.com/artpar/artpar/issues/4062), but there is a 52191 workaround: 52192 .IP "1." 3 52193 Visit https://onedrive.live.com (https://onedrive.live.com/) 52194 .IP "2." 3 52195 Right click a item in \f[C]Shared\f[R], then click 52196 \f[C]Add shortcut to My files\f[R] in the context 52197 [IMAGE: make_shortcut (https://user-images.githubusercontent.com/60313789/206118040-7e762b3b-aa61-41a1-8649-cc18889f3572.png)] 52198 .IP "3." 3 52199 The shortcut will appear in \f[C]My files\f[R], you can access it with 52200 rclone, it behaves like a normal folder/file. 52201 [IMAGE: in_my_files (https://i.imgur.com/0S8H3li.png)] 52202 [IMAGE: rclone_mount (https://i.imgur.com/2Iq66sW.png)] 52203 .SS Live Photos uploaded from iOS (small video clips in .heic files) 52204 .PP 52205 The iOS OneDrive app introduced upload and 52206 storage (https://techcommunity.microsoft.com/t5/microsoft-onedrive-blog/live-photos-come-to-onedrive/ba-p/1953452) 52207 of Live Photos (https://support.apple.com/en-gb/HT207310) in 2020. 52208 The usage and download of these uploaded Live Photos is unfortunately 52209 still work-in-progress and this introduces several issues when copying, 52210 synchronising and mounting \[en] both in rclone and in the native 52211 OneDrive client on Windows. 52212 .PP 52213 The root cause can easily be seen if you locate one of your Live Photos 52214 in the OneDrive web interface. 52215 Then download the photo from the web interface. 52216 You will then see that the size of downloaded .heic file is smaller than 52217 the size displayed in the web interface. 52218 The downloaded file is smaller because it only contains a single frame 52219 (still photo) extracted from the Live Photo (movie) stored in OneDrive. 52220 .PP 52221 The different sizes will cause \f[C]rclone copy/sync\f[R] to repeatedly 52222 recopy unmodified photos something like this: 52223 .IP 52224 .nf 52225 \f[C] 52226 DEBUG : 20230203_123826234_iOS.heic: Sizes differ (src 4470314 vs dst 1298667) 52227 DEBUG : 20230203_123826234_iOS.heic: sha1 = fc2edde7863b7a7c93ca6771498ac797f8460750 OK 52228 INFO : 20230203_123826234_iOS.heic: Copied (replaced existing) 52229 \f[R] 52230 .fi 52231 .PP 52232 These recopies can be worked around by adding \f[C]--ignore-size\f[R]. 52233 Please note that this workaround only syncs the still-picture not the 52234 movie clip, and relies on modification dates being correctly updated on 52235 all files in all situations. 52236 .PP 52237 The different sizes will also cause \f[C]rclone check\f[R] to report 52238 size errors something like this: 52239 .IP 52240 .nf 52241 \f[C] 52242 ERROR : 20230203_123826234_iOS.heic: sizes differ 52243 \f[R] 52244 .fi 52245 .PP 52246 These check errors can be suppressed by adding \f[C]--ignore-size\f[R]. 52247 .PP 52248 The different sizes will also cause \f[C]rclone mount\f[R] to fail 52249 downloading with an error something like this: 52250 .IP 52251 .nf 52252 \f[C] 52253 ERROR : 20230203_123826234_iOS.heic: ReadFileHandle.Read error: low level retry 1/10: unexpected EOF 52254 \f[R] 52255 .fi 52256 .PP 52257 or like this when using \f[C]--cache-mode=full\f[R]: 52258 .IP 52259 .nf 52260 \f[C] 52261 INFO : 20230203_123826234_iOS.heic: vfs cache: downloader: error count now 1: vfs reader: failed to write to cache file: 416 Requested Range Not Satisfiable: 52262 ERROR : 20230203_123826234_iOS.heic: vfs cache: failed to download: vfs reader: failed to write to cache file: 416 Requested Range Not Satisfiable: 52263 \f[R] 52264 .fi 52265 .SH OpenDrive 52266 .PP 52267 Paths are specified as \f[C]remote:path\f[R] 52268 .PP 52269 Paths may be as deep as required, e.g. 52270 \f[C]remote:directory/subdirectory\f[R]. 52271 .SS Configuration 52272 .PP 52273 Here is an example of how to make a remote called \f[C]remote\f[R]. 52274 First run: 52275 .IP 52276 .nf 52277 \f[C] 52278 rclone config 52279 \f[R] 52280 .fi 52281 .PP 52282 This will guide you through an interactive setup process: 52283 .IP 52284 .nf 52285 \f[C] 52286 n) New remote 52287 d) Delete remote 52288 q) Quit config 52289 e/n/d/q> n 52290 name> remote 52291 Type of storage to configure. 52292 Choose a number from below, or type in your own value 52293 [snip] 52294 XX / OpenDrive 52295 \[rs] \[dq]opendrive\[dq] 52296 [snip] 52297 Storage> opendrive 52298 Username 52299 username> 52300 Password 52301 y) Yes type in my own password 52302 g) Generate random password 52303 y/g> y 52304 Enter the password: 52305 password: 52306 Confirm the password: 52307 password: 52308 -------------------- 52309 [remote] 52310 username = 52311 password = *** ENCRYPTED *** 52312 -------------------- 52313 y) Yes this is OK 52314 e) Edit this remote 52315 d) Delete this remote 52316 y/e/d> y 52317 \f[R] 52318 .fi 52319 .PP 52320 List directories in top level of your OpenDrive 52321 .IP 52322 .nf 52323 \f[C] 52324 rclone lsd remote: 52325 \f[R] 52326 .fi 52327 .PP 52328 List all the files in your OpenDrive 52329 .IP 52330 .nf 52331 \f[C] 52332 rclone ls remote: 52333 \f[R] 52334 .fi 52335 .PP 52336 To copy a local directory to an OpenDrive directory called backup 52337 .IP 52338 .nf 52339 \f[C] 52340 rclone copy /home/source remote:backup 52341 \f[R] 52342 .fi 52343 .SS Modification times and hashes 52344 .PP 52345 OpenDrive allows modification times to be set on objects accurate to 1 52346 second. 52347 These will be used to detect whether objects need syncing or not. 52348 .PP 52349 The MD5 hash algorithm is supported. 52350 .SS Restricted filename characters 52351 .PP 52352 .TS 52353 tab(@); 52354 l c c. 52355 T{ 52356 Character 52357 T}@T{ 52358 Value 52359 T}@T{ 52360 Replacement 52361 T} 52362 _ 52363 T{ 52364 NUL 52365 T}@T{ 52366 0x00 52367 T}@T{ 52368 \[u2400] 52369 T} 52370 T{ 52371 / 52372 T}@T{ 52373 0x2F 52374 T}@T{ 52375 \[uFF0F] 52376 T} 52377 T{ 52378 \[dq] 52379 T}@T{ 52380 0x22 52381 T}@T{ 52382 \[uFF02] 52383 T} 52384 T{ 52385 * 52386 T}@T{ 52387 0x2A 52388 T}@T{ 52389 \[uFF0A] 52390 T} 52391 T{ 52392 : 52393 T}@T{ 52394 0x3A 52395 T}@T{ 52396 \[uFF1A] 52397 T} 52398 T{ 52399 < 52400 T}@T{ 52401 0x3C 52402 T}@T{ 52403 \[uFF1C] 52404 T} 52405 T{ 52406 > 52407 T}@T{ 52408 0x3E 52409 T}@T{ 52410 \[uFF1E] 52411 T} 52412 T{ 52413 ? 52414 T}@T{ 52415 0x3F 52416 T}@T{ 52417 \[uFF1F] 52418 T} 52419 T{ 52420 \[rs] 52421 T}@T{ 52422 0x5C 52423 T}@T{ 52424 \[uFF3C] 52425 T} 52426 T{ 52427 | 52428 T}@T{ 52429 0x7C 52430 T}@T{ 52431 \[uFF5C] 52432 T} 52433 .TE 52434 .PP 52435 File names can also not begin or end with the following characters. 52436 These only get replaced if they are the first or last character in the 52437 name: 52438 .PP 52439 .TS 52440 tab(@); 52441 l c c. 52442 T{ 52443 Character 52444 T}@T{ 52445 Value 52446 T}@T{ 52447 Replacement 52448 T} 52449 _ 52450 T{ 52451 SP 52452 T}@T{ 52453 0x20 52454 T}@T{ 52455 \[u2420] 52456 T} 52457 T{ 52458 HT 52459 T}@T{ 52460 0x09 52461 T}@T{ 52462 \[u2409] 52463 T} 52464 T{ 52465 LF 52466 T}@T{ 52467 0x0A 52468 T}@T{ 52469 \[u240A] 52470 T} 52471 T{ 52472 VT 52473 T}@T{ 52474 0x0B 52475 T}@T{ 52476 \[u240B] 52477 T} 52478 T{ 52479 CR 52480 T}@T{ 52481 0x0D 52482 T}@T{ 52483 \[u240D] 52484 T} 52485 .TE 52486 .PP 52487 Invalid UTF-8 bytes will also be 52488 replaced (https://rclone.org/overview/#invalid-utf8), as they can\[aq]t 52489 be used in JSON strings. 52490 .SS Standard options 52491 .PP 52492 Here are the Standard options specific to opendrive (OpenDrive). 52493 .SS --opendrive-username 52494 .PP 52495 Username. 52496 .PP 52497 Properties: 52498 .IP \[bu] 2 52499 Config: username 52500 .IP \[bu] 2 52501 Env Var: RCLONE_OPENDRIVE_USERNAME 52502 .IP \[bu] 2 52503 Type: string 52504 .IP \[bu] 2 52505 Required: true 52506 .SS --opendrive-password 52507 .PP 52508 Password. 52509 .PP 52510 \f[B]NB\f[R] Input to this must be obscured - see rclone 52511 obscure (https://rclone.org/commands/rclone_obscure/). 52512 .PP 52513 Properties: 52514 .IP \[bu] 2 52515 Config: password 52516 .IP \[bu] 2 52517 Env Var: RCLONE_OPENDRIVE_PASSWORD 52518 .IP \[bu] 2 52519 Type: string 52520 .IP \[bu] 2 52521 Required: true 52522 .SS Advanced options 52523 .PP 52524 Here are the Advanced options specific to opendrive (OpenDrive). 52525 .SS --opendrive-encoding 52526 .PP 52527 The encoding for the backend. 52528 .PP 52529 See the encoding section in the 52530 overview (https://rclone.org/overview/#encoding) for more info. 52531 .PP 52532 Properties: 52533 .IP \[bu] 2 52534 Config: encoding 52535 .IP \[bu] 2 52536 Env Var: RCLONE_OPENDRIVE_ENCODING 52537 .IP \[bu] 2 52538 Type: Encoding 52539 .IP \[bu] 2 52540 Default: 52541 Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,LeftSpace,LeftCrLfHtVt,RightSpace,RightCrLfHtVt,InvalidUtf8,Dot 52542 .SS --opendrive-chunk-size 52543 .PP 52544 Files will be uploaded in chunks this size. 52545 .PP 52546 Note that these chunks are buffered in memory so increasing them will 52547 increase memory use. 52548 .PP 52549 Properties: 52550 .IP \[bu] 2 52551 Config: chunk_size 52552 .IP \[bu] 2 52553 Env Var: RCLONE_OPENDRIVE_CHUNK_SIZE 52554 .IP \[bu] 2 52555 Type: SizeSuffix 52556 .IP \[bu] 2 52557 Default: 10Mi 52558 .SS --opendrive-description 52559 .PP 52560 Description of the remote 52561 .PP 52562 Properties: 52563 .IP \[bu] 2 52564 Config: description 52565 .IP \[bu] 2 52566 Env Var: RCLONE_OPENDRIVE_DESCRIPTION 52567 .IP \[bu] 2 52568 Type: string 52569 .IP \[bu] 2 52570 Required: false 52571 .SS Limitations 52572 .PP 52573 Note that OpenDrive is case insensitive so you can\[aq]t have a file 52574 called \[dq]Hello.doc\[dq] and one called \[dq]hello.doc\[dq]. 52575 .PP 52576 There are quite a few characters that can\[aq]t be in OpenDrive file 52577 names. 52578 These can\[aq]t occur on Windows platforms, but on non-Windows platforms 52579 they are common. 52580 Rclone will map these names to and from an identical looking unicode 52581 equivalent. 52582 For example if a file has a \f[C]?\f[R] in it will be mapped to 52583 \f[C]\[uFF1F]\f[R] instead. 52584 .PP 52585 \f[C]rclone about\f[R] is not supported by the OpenDrive backend. 52586 Backends without this capability cannot determine free space for an 52587 rclone mount or use policy \f[C]mfs\f[R] (most free space) as a member 52588 of an rclone union remote. 52589 .PP 52590 See List of backends that do not support rclone 52591 about (https://rclone.org/overview/#optional-features) and rclone 52592 about (https://rclone.org/commands/rclone_about/) 52593 .SH Oracle Object Storage 52594 .IP \[bu] 2 52595 Oracle Object Storage 52596 Overview (https://docs.oracle.com/en-us/iaas/Content/Object/Concepts/objectstorageoverview.htm) 52597 .IP \[bu] 2 52598 Oracle Object Storage 52599 FAQ (https://www.oracle.com/cloud/storage/object-storage/faq/) 52600 .IP \[bu] 2 52601 Oracle Object Storage 52602 Limits (https://docs.oracle.com/en-us/iaas/Content/Resources/Assets/whitepapers/oci-object-storage-best-practices.pdf) 52603 .PP 52604 Paths are specified as \f[C]remote:bucket\f[R] (or \f[C]remote:\f[R] for 52605 the \f[C]lsd\f[R] command.) You may put subdirectories in too, e.g. 52606 \f[C]remote:bucket/path/to/dir\f[R]. 52607 .PP 52608 Sample command to transfer local artifacts to remote:bucket in oracle 52609 object storage: 52610 .PP 52611 \f[C]rclone -vvv --progress --stats-one-line --max-stats-groups 10 --log-format date,time,UTC,longfile --fast-list --buffer-size 256Mi --oos-no-check-bucket --oos-upload-cutoff 10Mi --multi-thread-cutoff 16Mi --multi-thread-streams 3000 --transfers 3000 --checkers 64 --retries 2 --oos-chunk-size 10Mi --oos-upload-concurrency 10000 --oos-attempt-resume-upload --oos-leave-parts-on-error sync ./artifacts remote:bucket -vv\f[R] 52612 .SS Configuration 52613 .PP 52614 Here is an example of making an oracle object storage configuration. 52615 \f[C]rclone config\f[R] walks you through it. 52616 .PP 52617 Here is an example of how to make a remote called \f[C]remote\f[R]. 52618 First run: 52619 .IP 52620 .nf 52621 \f[C] 52622 rclone config 52623 \f[R] 52624 .fi 52625 .PP 52626 This will guide you through an interactive setup process: 52627 .IP 52628 .nf 52629 \f[C] 52630 n) New remote 52631 d) Delete remote 52632 r) Rename remote 52633 c) Copy remote 52634 s) Set configuration password 52635 q) Quit config 52636 e/n/d/r/c/s/q> n 52637 52638 Enter name for new remote. 52639 name> remote 52640 52641 Option Storage. 52642 Type of storage to configure. 52643 Choose a number from below, or type in your own value. 52644 [snip] 52645 XX / Oracle Cloud Infrastructure Object Storage 52646 \[rs] (oracleobjectstorage) 52647 Storage> oracleobjectstorage 52648 52649 Option provider. 52650 Choose your Auth Provider 52651 Choose a number from below, or type in your own string value. 52652 Press Enter for the default (env_auth). 52653 1 / automatically pickup the credentials from runtime(env), first one to provide auth wins 52654 \[rs] (env_auth) 52655 / use an OCI user and an API key for authentication. 52656 2 | you\[cq]ll need to put in a config file your tenancy OCID, user OCID, region, the path, fingerprint to an API key. 52657 | https://docs.oracle.com/en-us/iaas/Content/API/Concepts/sdkconfig.htm 52658 \[rs] (user_principal_auth) 52659 / use instance principals to authorize an instance to make API calls. 52660 3 | each instance has its own identity, and authenticates using the certificates that are read from instance metadata. 52661 | https://docs.oracle.com/en-us/iaas/Content/Identity/Tasks/callingservicesfrominstances.htm 52662 \[rs] (instance_principal_auth) 52663 / use workload identity to grant Kubernetes pods policy-driven access to Oracle Cloud 52664 4 | Infrastructure (OCI) resources using OCI Identity and Access Management (IAM). 52665 | https://docs.oracle.com/en-us/iaas/Content/ContEng/Tasks/contenggrantingworkloadaccesstoresources.htm 52666 \[rs] (workload_identity_auth) 52667 5 / use resource principals to make API calls 52668 \[rs] (resource_principal_auth) 52669 6 / no credentials needed, this is typically for reading public buckets 52670 \[rs] (no_auth) 52671 provider> 2 52672 52673 Option namespace. 52674 Object storage namespace 52675 Enter a value. 52676 namespace> idbamagbg734 52677 52678 Option compartment. 52679 Object storage compartment OCID 52680 Enter a value. 52681 compartment> ocid1.compartment.oc1..aaaaaaaapufkxc7ame3sthry5i7ujrwfc7ejnthhu6bhanm5oqfjpyasjkba 52682 52683 Option region. 52684 Object storage Region 52685 Enter a value. 52686 region> us-ashburn-1 52687 52688 Option endpoint. 52689 Endpoint for Object storage API. 52690 Leave blank to use the default endpoint for the region. 52691 Enter a value. Press Enter to leave empty. 52692 endpoint> 52693 52694 Option config_file. 52695 Full Path to OCI config file 52696 Choose a number from below, or type in your own string value. 52697 Press Enter for the default (\[ti]/.oci/config). 52698 1 / oci configuration file location 52699 \[rs] (\[ti]/.oci/config) 52700 config_file> /etc/oci/dev.conf 52701 52702 Option config_profile. 52703 Profile name inside OCI config file 52704 Choose a number from below, or type in your own string value. 52705 Press Enter for the default (Default). 52706 1 / Use the default profile 52707 \[rs] (Default) 52708 config_profile> Test 52709 52710 Edit advanced config? 52711 y) Yes 52712 n) No (default) 52713 y/n> n 52714 52715 Configuration complete. 52716 Options: 52717 - type: oracleobjectstorage 52718 - namespace: idbamagbg734 52719 - compartment: ocid1.compartment.oc1..aaaaaaaapufkxc7ame3sthry5i7ujrwfc7ejnthhu6bhanm5oqfjpyasjkba 52720 - region: us-ashburn-1 52721 - provider: user_principal_auth 52722 - config_file: /etc/oci/dev.conf 52723 - config_profile: Test 52724 Keep this \[dq]remote\[dq] remote? 52725 y) Yes this is OK (default) 52726 e) Edit this remote 52727 d) Delete this remote 52728 y/e/d> y 52729 \f[R] 52730 .fi 52731 .PP 52732 See all buckets 52733 .IP 52734 .nf 52735 \f[C] 52736 rclone lsd remote: 52737 \f[R] 52738 .fi 52739 .PP 52740 Create a new bucket 52741 .IP 52742 .nf 52743 \f[C] 52744 rclone mkdir remote:bucket 52745 \f[R] 52746 .fi 52747 .PP 52748 List the contents of a bucket 52749 .IP 52750 .nf 52751 \f[C] 52752 rclone ls remote:bucket 52753 rclone ls remote:bucket --max-depth 1 52754 \f[R] 52755 .fi 52756 .SS Authentication Providers 52757 .PP 52758 OCI has various authentication methods. 52759 To learn more about authentication methods please refer oci 52760 authentication 52761 methods (https://docs.oracle.com/en-us/iaas/Content/API/Concepts/sdk_authentication_methods.htm) 52762 These choices can be specified in the rclone config file. 52763 .PP 52764 Rclone supports the following OCI authentication provider. 52765 .IP 52766 .nf 52767 \f[C] 52768 User Principal 52769 Instance Principal 52770 Resource Principal 52771 Workload Identity 52772 No authentication 52773 \f[R] 52774 .fi 52775 .SS User Principal 52776 .PP 52777 Sample rclone config file for Authentication Provider User Principal: 52778 .IP 52779 .nf 52780 \f[C] 52781 [oos] 52782 type = oracleobjectstorage 52783 namespace = id<redacted>34 52784 compartment = ocid1.compartment.oc1..aa<redacted>ba 52785 region = us-ashburn-1 52786 provider = user_principal_auth 52787 config_file = /home/opc/.oci/config 52788 config_profile = Default 52789 \f[R] 52790 .fi 52791 .PP 52792 Advantages: - One can use this method from any server within OCI or 52793 on-premises or from other cloud provider. 52794 .PP 52795 Considerations: - you need to configure user\[cq]s privileges / policy 52796 to allow access to object storage - Overhead of managing users and keys. 52797 - If the user is deleted, the config file will no longer work and may 52798 cause automation regressions that use the user\[aq]s credentials. 52799 .SS Instance Principal 52800 .PP 52801 An OCI compute instance can be authorized to use rclone by using 52802 it\[aq]s identity and certificates as an instance principal. 52803 With this approach no credentials have to be stored and managed. 52804 .PP 52805 Sample rclone configuration file for Authentication Provider Instance 52806 Principal: 52807 .IP 52808 .nf 52809 \f[C] 52810 [opc\[at]rclone \[ti]]$ cat \[ti]/.config/artpar/artpar.conf 52811 [oos] 52812 type = oracleobjectstorage 52813 namespace = id<redacted>fn 52814 compartment = ocid1.compartment.oc1..aa<redacted>k7a 52815 region = us-ashburn-1 52816 provider = instance_principal_auth 52817 \f[R] 52818 .fi 52819 .PP 52820 Advantages: 52821 .IP \[bu] 2 52822 With instance principals, you don\[aq]t need to configure user 52823 credentials and transfer/ save it to disk in your compute instances or 52824 rotate the credentials. 52825 .IP \[bu] 2 52826 You don\[cq]t need to deal with users and keys. 52827 .IP \[bu] 2 52828 Greatly helps in automation as you don\[aq]t have to manage access keys, 52829 user private keys, storing them in vault, using kms etc. 52830 .PP 52831 Considerations: 52832 .IP \[bu] 2 52833 You need to configure a dynamic group having this instance as member and 52834 add policy to read object storage to that dynamic group. 52835 .IP \[bu] 2 52836 Everyone who has access to this machine can execute the CLI commands. 52837 .IP \[bu] 2 52838 It is applicable for oci compute instances only. 52839 It cannot be used on external instance or resources. 52840 .SS Resource Principal 52841 .PP 52842 Resource principal auth is very similar to instance principal auth but 52843 used for resources that are not compute instances such as serverless 52844 functions (https://docs.oracle.com/en-us/iaas/Content/Functions/Concepts/functionsoverview.htm). 52845 To use resource principal ensure Rclone process is started with these 52846 environment variables set in its process. 52847 .IP 52848 .nf 52849 \f[C] 52850 export OCI_RESOURCE_PRINCIPAL_VERSION=2.2 52851 export OCI_RESOURCE_PRINCIPAL_REGION=us-ashburn-1 52852 export OCI_RESOURCE_PRINCIPAL_PRIVATE_PEM=/usr/share/model-server/key.pem 52853 export OCI_RESOURCE_PRINCIPAL_RPST=/usr/share/model-server/security_token 52854 \f[R] 52855 .fi 52856 .PP 52857 Sample rclone configuration file for Authentication Provider Resource 52858 Principal: 52859 .IP 52860 .nf 52861 \f[C] 52862 [oos] 52863 type = oracleobjectstorage 52864 namespace = id<redacted>34 52865 compartment = ocid1.compartment.oc1..aa<redacted>ba 52866 region = us-ashburn-1 52867 provider = resource_principal_auth 52868 \f[R] 52869 .fi 52870 .SS Workload Identity 52871 .PP 52872 Workload Identity auth may be used when running Rclone from Kubernetes 52873 pod on a Container Engine for Kubernetes (OKE) cluster. 52874 For more details on configuring Workload Identity, see Granting 52875 Workloads Access to OCI 52876 Resources (https://docs.oracle.com/en-us/iaas/Content/ContEng/Tasks/contenggrantingworkloadaccesstoresources.htm). 52877 To use workload identity, ensure Rclone is started with these 52878 environment variables set in its process. 52879 .IP 52880 .nf 52881 \f[C] 52882 export OCI_RESOURCE_PRINCIPAL_VERSION=2.2 52883 export OCI_RESOURCE_PRINCIPAL_REGION=us-ashburn-1 52884 \f[R] 52885 .fi 52886 .SS No authentication 52887 .PP 52888 Public buckets do not require any authentication mechanism to read 52889 objects. 52890 Sample rclone configuration file for No authentication: 52891 .IP 52892 .nf 52893 \f[C] 52894 [oos] 52895 type = oracleobjectstorage 52896 namespace = id<redacted>34 52897 compartment = ocid1.compartment.oc1..aa<redacted>ba 52898 region = us-ashburn-1 52899 provider = no_auth 52900 \f[R] 52901 .fi 52902 .SS Modification times and hashes 52903 .PP 52904 The modification time is stored as metadata on the object as 52905 \f[C]opc-meta-mtime\f[R] as floating point since the epoch, accurate to 52906 1 ns. 52907 .PP 52908 If the modification time needs to be updated rclone will attempt to 52909 perform a server side copy to update the modification if the object can 52910 be copied in a single part. 52911 In the case the object is larger than 5Gb, the object will be uploaded 52912 rather than copied. 52913 .PP 52914 Note that reading this from the object takes an additional 52915 \f[C]HEAD\f[R] request as the metadata isn\[aq]t returned in object 52916 listings. 52917 .PP 52918 The MD5 hash algorithm is supported. 52919 .SS Multipart uploads 52920 .PP 52921 rclone supports multipart uploads with OOS which means that it can 52922 upload files bigger than 5 GiB. 52923 .PP 52924 Note that files uploaded \f[I]both\f[R] with multipart upload 52925 \f[I]and\f[R] through crypt remotes do not have MD5 sums. 52926 .PP 52927 rclone switches from single part uploads to multipart uploads at the 52928 point specified by \f[C]--oos-upload-cutoff\f[R]. 52929 This can be a maximum of 5 GiB and a minimum of 0 (ie always upload 52930 multipart files). 52931 .PP 52932 The chunk sizes used in the multipart upload are specified by 52933 \f[C]--oos-chunk-size\f[R] and the number of chunks uploaded 52934 concurrently is specified by \f[C]--oos-upload-concurrency\f[R]. 52935 .PP 52936 Multipart uploads will use \f[C]--transfers\f[R] * 52937 \f[C]--oos-upload-concurrency\f[R] * \f[C]--oos-chunk-size\f[R] extra 52938 memory. 52939 Single part uploads to not use extra memory. 52940 .PP 52941 Single part transfers can be faster than multipart transfers or slower 52942 depending on your latency from oos - the more latency, the more likely 52943 single part transfers will be faster. 52944 .PP 52945 Increasing \f[C]--oos-upload-concurrency\f[R] will increase throughput 52946 (8 would be a sensible value) and increasing \f[C]--oos-chunk-size\f[R] 52947 also increases throughput (16M would be sensible). 52948 Increasing either of these will use more memory. 52949 The default values are high enough to gain most of the possible 52950 performance without using too much memory. 52951 .SS Standard options 52952 .PP 52953 Here are the Standard options specific to oracleobjectstorage (Oracle 52954 Cloud Infrastructure Object Storage). 52955 .SS --oos-provider 52956 .PP 52957 Choose your Auth Provider 52958 .PP 52959 Properties: 52960 .IP \[bu] 2 52961 Config: provider 52962 .IP \[bu] 2 52963 Env Var: RCLONE_OOS_PROVIDER 52964 .IP \[bu] 2 52965 Type: string 52966 .IP \[bu] 2 52967 Default: \[dq]env_auth\[dq] 52968 .IP \[bu] 2 52969 Examples: 52970 .RS 2 52971 .IP \[bu] 2 52972 \[dq]env_auth\[dq] 52973 .RS 2 52974 .IP \[bu] 2 52975 automatically pickup the credentials from runtime(env), first one to 52976 provide auth wins 52977 .RE 52978 .IP \[bu] 2 52979 \[dq]user_principal_auth\[dq] 52980 .RS 2 52981 .IP \[bu] 2 52982 use an OCI user and an API key for authentication. 52983 .IP \[bu] 2 52984 you\[cq]ll need to put in a config file your tenancy OCID, user OCID, 52985 region, the path, fingerprint to an API key. 52986 .IP \[bu] 2 52987 https://docs.oracle.com/en-us/iaas/Content/API/Concepts/sdkconfig.htm 52988 .RE 52989 .IP \[bu] 2 52990 \[dq]instance_principal_auth\[dq] 52991 .RS 2 52992 .IP \[bu] 2 52993 use instance principals to authorize an instance to make API calls. 52994 .IP \[bu] 2 52995 each instance has its own identity, and authenticates using the 52996 certificates that are read from instance metadata. 52997 .IP \[bu] 2 52998 https://docs.oracle.com/en-us/iaas/Content/Identity/Tasks/callingservicesfrominstances.htm 52999 .RE 53000 .IP \[bu] 2 53001 \[dq]workload_identity_auth\[dq] 53002 .RS 2 53003 .IP \[bu] 2 53004 use workload identity to grant OCI Container Engine for Kubernetes 53005 workloads policy-driven access to OCI resources using OCI Identity and 53006 Access Management (IAM). 53007 .IP \[bu] 2 53008 https://docs.oracle.com/en-us/iaas/Content/ContEng/Tasks/contenggrantingworkloadaccesstoresources.htm 53009 .RE 53010 .IP \[bu] 2 53011 \[dq]resource_principal_auth\[dq] 53012 .RS 2 53013 .IP \[bu] 2 53014 use resource principals to make API calls 53015 .RE 53016 .IP \[bu] 2 53017 \[dq]no_auth\[dq] 53018 .RS 2 53019 .IP \[bu] 2 53020 no credentials needed, this is typically for reading public buckets 53021 .RE 53022 .RE 53023 .SS --oos-namespace 53024 .PP 53025 Object storage namespace 53026 .PP 53027 Properties: 53028 .IP \[bu] 2 53029 Config: namespace 53030 .IP \[bu] 2 53031 Env Var: RCLONE_OOS_NAMESPACE 53032 .IP \[bu] 2 53033 Type: string 53034 .IP \[bu] 2 53035 Required: true 53036 .SS --oos-compartment 53037 .PP 53038 Object storage compartment OCID 53039 .PP 53040 Properties: 53041 .IP \[bu] 2 53042 Config: compartment 53043 .IP \[bu] 2 53044 Env Var: RCLONE_OOS_COMPARTMENT 53045 .IP \[bu] 2 53046 Provider: !no_auth 53047 .IP \[bu] 2 53048 Type: string 53049 .IP \[bu] 2 53050 Required: true 53051 .SS --oos-region 53052 .PP 53053 Object storage Region 53054 .PP 53055 Properties: 53056 .IP \[bu] 2 53057 Config: region 53058 .IP \[bu] 2 53059 Env Var: RCLONE_OOS_REGION 53060 .IP \[bu] 2 53061 Type: string 53062 .IP \[bu] 2 53063 Required: true 53064 .SS --oos-endpoint 53065 .PP 53066 Endpoint for Object storage API. 53067 .PP 53068 Leave blank to use the default endpoint for the region. 53069 .PP 53070 Properties: 53071 .IP \[bu] 2 53072 Config: endpoint 53073 .IP \[bu] 2 53074 Env Var: RCLONE_OOS_ENDPOINT 53075 .IP \[bu] 2 53076 Type: string 53077 .IP \[bu] 2 53078 Required: false 53079 .SS --oos-config-file 53080 .PP 53081 Path to OCI config file 53082 .PP 53083 Properties: 53084 .IP \[bu] 2 53085 Config: config_file 53086 .IP \[bu] 2 53087 Env Var: RCLONE_OOS_CONFIG_FILE 53088 .IP \[bu] 2 53089 Provider: user_principal_auth 53090 .IP \[bu] 2 53091 Type: string 53092 .IP \[bu] 2 53093 Default: \[dq]\[ti]/.oci/config\[dq] 53094 .IP \[bu] 2 53095 Examples: 53096 .RS 2 53097 .IP \[bu] 2 53098 \[dq]\[ti]/.oci/config\[dq] 53099 .RS 2 53100 .IP \[bu] 2 53101 oci configuration file location 53102 .RE 53103 .RE 53104 .SS --oos-config-profile 53105 .PP 53106 Profile name inside the oci config file 53107 .PP 53108 Properties: 53109 .IP \[bu] 2 53110 Config: config_profile 53111 .IP \[bu] 2 53112 Env Var: RCLONE_OOS_CONFIG_PROFILE 53113 .IP \[bu] 2 53114 Provider: user_principal_auth 53115 .IP \[bu] 2 53116 Type: string 53117 .IP \[bu] 2 53118 Default: \[dq]Default\[dq] 53119 .IP \[bu] 2 53120 Examples: 53121 .RS 2 53122 .IP \[bu] 2 53123 \[dq]Default\[dq] 53124 .RS 2 53125 .IP \[bu] 2 53126 Use the default profile 53127 .RE 53128 .RE 53129 .SS Advanced options 53130 .PP 53131 Here are the Advanced options specific to oracleobjectstorage (Oracle 53132 Cloud Infrastructure Object Storage). 53133 .SS --oos-storage-tier 53134 .PP 53135 The storage class to use when storing new objects in storage. 53136 https://docs.oracle.com/en-us/iaas/Content/Object/Concepts/understandingstoragetiers.htm 53137 .PP 53138 Properties: 53139 .IP \[bu] 2 53140 Config: storage_tier 53141 .IP \[bu] 2 53142 Env Var: RCLONE_OOS_STORAGE_TIER 53143 .IP \[bu] 2 53144 Type: string 53145 .IP \[bu] 2 53146 Default: \[dq]Standard\[dq] 53147 .IP \[bu] 2 53148 Examples: 53149 .RS 2 53150 .IP \[bu] 2 53151 \[dq]Standard\[dq] 53152 .RS 2 53153 .IP \[bu] 2 53154 Standard storage tier, this is the default tier 53155 .RE 53156 .IP \[bu] 2 53157 \[dq]InfrequentAccess\[dq] 53158 .RS 2 53159 .IP \[bu] 2 53160 InfrequentAccess storage tier 53161 .RE 53162 .IP \[bu] 2 53163 \[dq]Archive\[dq] 53164 .RS 2 53165 .IP \[bu] 2 53166 Archive storage tier 53167 .RE 53168 .RE 53169 .SS --oos-upload-cutoff 53170 .PP 53171 Cutoff for switching to chunked upload. 53172 .PP 53173 Any files larger than this will be uploaded in chunks of chunk_size. 53174 The minimum is 0 and the maximum is 5 GiB. 53175 .PP 53176 Properties: 53177 .IP \[bu] 2 53178 Config: upload_cutoff 53179 .IP \[bu] 2 53180 Env Var: RCLONE_OOS_UPLOAD_CUTOFF 53181 .IP \[bu] 2 53182 Type: SizeSuffix 53183 .IP \[bu] 2 53184 Default: 200Mi 53185 .SS --oos-chunk-size 53186 .PP 53187 Chunk size to use for uploading. 53188 .PP 53189 When uploading files larger than upload_cutoff or files with unknown 53190 size (e.g. 53191 from \[dq]rclone rcat\[dq] or uploaded with \[dq]rclone mount\[dq] they 53192 will be uploaded as multipart uploads using this chunk size. 53193 .PP 53194 Note that \[dq]upload_concurrency\[dq] chunks of this size are buffered 53195 in memory per transfer. 53196 .PP 53197 If you are transferring large files over high-speed links and you have 53198 enough memory, then increasing this will speed up the transfers. 53199 .PP 53200 Rclone will automatically increase the chunk size when uploading a large 53201 file of known size to stay below the 10,000 chunks limit. 53202 .PP 53203 Files of unknown size are uploaded with the configured chunk_size. 53204 Since the default chunk size is 5 MiB and there can be at most 10,000 53205 chunks, this means that by default the maximum size of a file you can 53206 stream upload is 48 GiB. 53207 If you wish to stream upload larger files then you will need to increase 53208 chunk_size. 53209 .PP 53210 Increasing the chunk size decreases the accuracy of the progress 53211 statistics displayed with \[dq]-P\[dq] flag. 53212 .PP 53213 Properties: 53214 .IP \[bu] 2 53215 Config: chunk_size 53216 .IP \[bu] 2 53217 Env Var: RCLONE_OOS_CHUNK_SIZE 53218 .IP \[bu] 2 53219 Type: SizeSuffix 53220 .IP \[bu] 2 53221 Default: 5Mi 53222 .SS --oos-max-upload-parts 53223 .PP 53224 Maximum number of parts in a multipart upload. 53225 .PP 53226 This option defines the maximum number of multipart chunks to use when 53227 doing a multipart upload. 53228 .PP 53229 OCI has max parts limit of 10,000 chunks. 53230 .PP 53231 Rclone will automatically increase the chunk size when uploading a large 53232 file of a known size to stay below this number of chunks limit. 53233 .PP 53234 Properties: 53235 .IP \[bu] 2 53236 Config: max_upload_parts 53237 .IP \[bu] 2 53238 Env Var: RCLONE_OOS_MAX_UPLOAD_PARTS 53239 .IP \[bu] 2 53240 Type: int 53241 .IP \[bu] 2 53242 Default: 10000 53243 .SS --oos-upload-concurrency 53244 .PP 53245 Concurrency for multipart uploads. 53246 .PP 53247 This is the number of chunks of the same file that are uploaded 53248 concurrently. 53249 .PP 53250 If you are uploading small numbers of large files over high-speed links 53251 and these uploads do not fully utilize your bandwidth, then increasing 53252 this may help to speed up the transfers. 53253 .PP 53254 Properties: 53255 .IP \[bu] 2 53256 Config: upload_concurrency 53257 .IP \[bu] 2 53258 Env Var: RCLONE_OOS_UPLOAD_CONCURRENCY 53259 .IP \[bu] 2 53260 Type: int 53261 .IP \[bu] 2 53262 Default: 10 53263 .SS --oos-copy-cutoff 53264 .PP 53265 Cutoff for switching to multipart copy. 53266 .PP 53267 Any files larger than this that need to be server-side copied will be 53268 copied in chunks of this size. 53269 .PP 53270 The minimum is 0 and the maximum is 5 GiB. 53271 .PP 53272 Properties: 53273 .IP \[bu] 2 53274 Config: copy_cutoff 53275 .IP \[bu] 2 53276 Env Var: RCLONE_OOS_COPY_CUTOFF 53277 .IP \[bu] 2 53278 Type: SizeSuffix 53279 .IP \[bu] 2 53280 Default: 4.656Gi 53281 .SS --oos-copy-timeout 53282 .PP 53283 Timeout for copy. 53284 .PP 53285 Copy is an asynchronous operation, specify timeout to wait for copy to 53286 succeed 53287 .PP 53288 Properties: 53289 .IP \[bu] 2 53290 Config: copy_timeout 53291 .IP \[bu] 2 53292 Env Var: RCLONE_OOS_COPY_TIMEOUT 53293 .IP \[bu] 2 53294 Type: Duration 53295 .IP \[bu] 2 53296 Default: 1m0s 53297 .SS --oos-disable-checksum 53298 .PP 53299 Don\[aq]t store MD5 checksum with object metadata. 53300 .PP 53301 Normally rclone will calculate the MD5 checksum of the input before 53302 uploading it so it can add it to metadata on the object. 53303 This is great for data integrity checking but can cause long delays for 53304 large files to start uploading. 53305 .PP 53306 Properties: 53307 .IP \[bu] 2 53308 Config: disable_checksum 53309 .IP \[bu] 2 53310 Env Var: RCLONE_OOS_DISABLE_CHECKSUM 53311 .IP \[bu] 2 53312 Type: bool 53313 .IP \[bu] 2 53314 Default: false 53315 .SS --oos-encoding 53316 .PP 53317 The encoding for the backend. 53318 .PP 53319 See the encoding section in the 53320 overview (https://rclone.org/overview/#encoding) for more info. 53321 .PP 53322 Properties: 53323 .IP \[bu] 2 53324 Config: encoding 53325 .IP \[bu] 2 53326 Env Var: RCLONE_OOS_ENCODING 53327 .IP \[bu] 2 53328 Type: Encoding 53329 .IP \[bu] 2 53330 Default: Slash,InvalidUtf8,Dot 53331 .SS --oos-leave-parts-on-error 53332 .PP 53333 If true avoid calling abort upload on a failure, leaving all 53334 successfully uploaded parts for manual recovery. 53335 .PP 53336 It should be set to true for resuming uploads across different sessions. 53337 .PP 53338 WARNING: Storing parts of an incomplete multipart upload counts towards 53339 space usage on object storage and will add additional costs if not 53340 cleaned up. 53341 .PP 53342 Properties: 53343 .IP \[bu] 2 53344 Config: leave_parts_on_error 53345 .IP \[bu] 2 53346 Env Var: RCLONE_OOS_LEAVE_PARTS_ON_ERROR 53347 .IP \[bu] 2 53348 Type: bool 53349 .IP \[bu] 2 53350 Default: false 53351 .SS --oos-attempt-resume-upload 53352 .PP 53353 If true attempt to resume previously started multipart upload for the 53354 object. 53355 This will be helpful to speed up multipart transfers by resuming uploads 53356 from past session. 53357 .PP 53358 WARNING: If chunk size differs in resumed session from past incomplete 53359 session, then the resumed multipart upload is aborted and a new 53360 multipart upload is started with the new chunk size. 53361 .PP 53362 The flag leave_parts_on_error must be true to resume and optimize to 53363 skip parts that were already uploaded successfully. 53364 .PP 53365 Properties: 53366 .IP \[bu] 2 53367 Config: attempt_resume_upload 53368 .IP \[bu] 2 53369 Env Var: RCLONE_OOS_ATTEMPT_RESUME_UPLOAD 53370 .IP \[bu] 2 53371 Type: bool 53372 .IP \[bu] 2 53373 Default: false 53374 .SS --oos-no-check-bucket 53375 .PP 53376 If set, don\[aq]t attempt to check the bucket exists or create it. 53377 .PP 53378 This can be useful when trying to minimise the number of transactions 53379 rclone does if you know the bucket exists already. 53380 .PP 53381 It can also be needed if the user you are using does not have bucket 53382 creation permissions. 53383 .PP 53384 Properties: 53385 .IP \[bu] 2 53386 Config: no_check_bucket 53387 .IP \[bu] 2 53388 Env Var: RCLONE_OOS_NO_CHECK_BUCKET 53389 .IP \[bu] 2 53390 Type: bool 53391 .IP \[bu] 2 53392 Default: false 53393 .SS --oos-sse-customer-key-file 53394 .PP 53395 To use SSE-C, a file containing the base64-encoded string of the AES-256 53396 encryption key associated with the object. 53397 Please note only one of 53398 sse_customer_key_file|sse_customer_key|sse_kms_key_id is needed.\[aq] 53399 .PP 53400 Properties: 53401 .IP \[bu] 2 53402 Config: sse_customer_key_file 53403 .IP \[bu] 2 53404 Env Var: RCLONE_OOS_SSE_CUSTOMER_KEY_FILE 53405 .IP \[bu] 2 53406 Type: string 53407 .IP \[bu] 2 53408 Required: false 53409 .IP \[bu] 2 53410 Examples: 53411 .RS 2 53412 .IP \[bu] 2 53413 \[dq]\[dq] 53414 .RS 2 53415 .IP \[bu] 2 53416 None 53417 .RE 53418 .RE 53419 .SS --oos-sse-customer-key 53420 .PP 53421 To use SSE-C, the optional header that specifies the base64-encoded 53422 256-bit encryption key to use to encrypt or decrypt the data. 53423 Please note only one of 53424 sse_customer_key_file|sse_customer_key|sse_kms_key_id is needed. 53425 For more information, see Using Your Own Keys for Server-Side Encryption 53426 (https://docs.cloud.oracle.com/Content/Object/Tasks/usingyourencryptionkeys.htm) 53427 .PP 53428 Properties: 53429 .IP \[bu] 2 53430 Config: sse_customer_key 53431 .IP \[bu] 2 53432 Env Var: RCLONE_OOS_SSE_CUSTOMER_KEY 53433 .IP \[bu] 2 53434 Type: string 53435 .IP \[bu] 2 53436 Required: false 53437 .IP \[bu] 2 53438 Examples: 53439 .RS 2 53440 .IP \[bu] 2 53441 \[dq]\[dq] 53442 .RS 2 53443 .IP \[bu] 2 53444 None 53445 .RE 53446 .RE 53447 .SS --oos-sse-customer-key-sha256 53448 .PP 53449 If using SSE-C, The optional header that specifies the base64-encoded 53450 SHA256 hash of the encryption key. 53451 This value is used to check the integrity of the encryption key. 53452 see Using Your Own Keys for Server-Side Encryption 53453 (https://docs.cloud.oracle.com/Content/Object/Tasks/usingyourencryptionkeys.htm). 53454 .PP 53455 Properties: 53456 .IP \[bu] 2 53457 Config: sse_customer_key_sha256 53458 .IP \[bu] 2 53459 Env Var: RCLONE_OOS_SSE_CUSTOMER_KEY_SHA256 53460 .IP \[bu] 2 53461 Type: string 53462 .IP \[bu] 2 53463 Required: false 53464 .IP \[bu] 2 53465 Examples: 53466 .RS 2 53467 .IP \[bu] 2 53468 \[dq]\[dq] 53469 .RS 2 53470 .IP \[bu] 2 53471 None 53472 .RE 53473 .RE 53474 .SS --oos-sse-kms-key-id 53475 .PP 53476 if using your own master key in vault, this header specifies the OCID 53477 (https://docs.cloud.oracle.com/Content/General/Concepts/identifiers.htm) 53478 of a master encryption key used to call the Key Management service to 53479 generate a data encryption key or to encrypt or decrypt a data 53480 encryption key. 53481 Please note only one of 53482 sse_customer_key_file|sse_customer_key|sse_kms_key_id is needed. 53483 .PP 53484 Properties: 53485 .IP \[bu] 2 53486 Config: sse_kms_key_id 53487 .IP \[bu] 2 53488 Env Var: RCLONE_OOS_SSE_KMS_KEY_ID 53489 .IP \[bu] 2 53490 Type: string 53491 .IP \[bu] 2 53492 Required: false 53493 .IP \[bu] 2 53494 Examples: 53495 .RS 2 53496 .IP \[bu] 2 53497 \[dq]\[dq] 53498 .RS 2 53499 .IP \[bu] 2 53500 None 53501 .RE 53502 .RE 53503 .SS --oos-sse-customer-algorithm 53504 .PP 53505 If using SSE-C, the optional header that specifies \[dq]AES256\[dq] as 53506 the encryption algorithm. 53507 Object Storage supports \[dq]AES256\[dq] as the encryption algorithm. 53508 For more information, see Using Your Own Keys for Server-Side Encryption 53509 (https://docs.cloud.oracle.com/Content/Object/Tasks/usingyourencryptionkeys.htm). 53510 .PP 53511 Properties: 53512 .IP \[bu] 2 53513 Config: sse_customer_algorithm 53514 .IP \[bu] 2 53515 Env Var: RCLONE_OOS_SSE_CUSTOMER_ALGORITHM 53516 .IP \[bu] 2 53517 Type: string 53518 .IP \[bu] 2 53519 Required: false 53520 .IP \[bu] 2 53521 Examples: 53522 .RS 2 53523 .IP \[bu] 2 53524 \[dq]\[dq] 53525 .RS 2 53526 .IP \[bu] 2 53527 None 53528 .RE 53529 .IP \[bu] 2 53530 \[dq]AES256\[dq] 53531 .RS 2 53532 .IP \[bu] 2 53533 AES256 53534 .RE 53535 .RE 53536 .SS --oos-description 53537 .PP 53538 Description of the remote 53539 .PP 53540 Properties: 53541 .IP \[bu] 2 53542 Config: description 53543 .IP \[bu] 2 53544 Env Var: RCLONE_OOS_DESCRIPTION 53545 .IP \[bu] 2 53546 Type: string 53547 .IP \[bu] 2 53548 Required: false 53549 .SS Backend commands 53550 .PP 53551 Here are the commands specific to the oracleobjectstorage backend. 53552 .PP 53553 Run them with 53554 .IP 53555 .nf 53556 \f[C] 53557 rclone backend COMMAND remote: 53558 \f[R] 53559 .fi 53560 .PP 53561 The help below will explain what arguments each command takes. 53562 .PP 53563 See the backend (https://rclone.org/commands/rclone_backend/) command 53564 for more info on how to pass options and arguments. 53565 .PP 53566 These can be run on a running backend using the rc command 53567 backend/command (https://rclone.org/rc/#backend-command). 53568 .SS rename 53569 .PP 53570 change the name of an object 53571 .IP 53572 .nf 53573 \f[C] 53574 rclone backend rename remote: [options] [<arguments>+] 53575 \f[R] 53576 .fi 53577 .PP 53578 This command can be used to rename a object. 53579 .PP 53580 Usage Examples: 53581 .IP 53582 .nf 53583 \f[C] 53584 rclone backend rename oos:bucket relative-object-path-under-bucket object-new-name 53585 \f[R] 53586 .fi 53587 .SS list-multipart-uploads 53588 .PP 53589 List the unfinished multipart uploads 53590 .IP 53591 .nf 53592 \f[C] 53593 rclone backend list-multipart-uploads remote: [options] [<arguments>+] 53594 \f[R] 53595 .fi 53596 .PP 53597 This command lists the unfinished multipart uploads in JSON format. 53598 .IP 53599 .nf 53600 \f[C] 53601 rclone backend list-multipart-uploads oos:bucket/path/to/object 53602 \f[R] 53603 .fi 53604 .PP 53605 It returns a dictionary of buckets with values as lists of unfinished 53606 multipart uploads. 53607 .PP 53608 You can call it with no bucket in which case it lists all bucket, with a 53609 bucket or with a bucket and path. 53610 .IP 53611 .nf 53612 \f[C] 53613 { 53614 \[dq]test-bucket\[dq]: [ 53615 { 53616 \[dq]namespace\[dq]: \[dq]test-namespace\[dq], 53617 \[dq]bucket\[dq]: \[dq]test-bucket\[dq], 53618 \[dq]object\[dq]: \[dq]600m.bin\[dq], 53619 \[dq]uploadId\[dq]: \[dq]51dd8114-52a4-b2f2-c42f-5291f05eb3c8\[dq], 53620 \[dq]timeCreated\[dq]: \[dq]2022-07-29T06:21:16.595Z\[dq], 53621 \[dq]storageTier\[dq]: \[dq]Standard\[dq] 53622 } 53623 ] 53624 \f[R] 53625 .fi 53626 .SS cleanup 53627 .PP 53628 Remove unfinished multipart uploads. 53629 .IP 53630 .nf 53631 \f[C] 53632 rclone backend cleanup remote: [options] [<arguments>+] 53633 \f[R] 53634 .fi 53635 .PP 53636 This command removes unfinished multipart uploads of age greater than 53637 max-age which defaults to 24 hours. 53638 .PP 53639 Note that you can use --interactive/-i or --dry-run with this command to 53640 see what it would do. 53641 .IP 53642 .nf 53643 \f[C] 53644 rclone backend cleanup oos:bucket/path/to/object 53645 rclone backend cleanup -o max-age=7w oos:bucket/path/to/object 53646 \f[R] 53647 .fi 53648 .PP 53649 Durations are parsed as per the rest of rclone, 2h, 7d, 7w etc. 53650 .PP 53651 Options: 53652 .IP \[bu] 2 53653 \[dq]max-age\[dq]: Max age of upload to delete 53654 .SS restore 53655 .PP 53656 Restore objects from Archive to Standard storage 53657 .IP 53658 .nf 53659 \f[C] 53660 rclone backend restore remote: [options] [<arguments>+] 53661 \f[R] 53662 .fi 53663 .PP 53664 This command can be used to restore one or more objects from Archive to 53665 Standard storage. 53666 .IP 53667 .nf 53668 \f[C] 53669 Usage Examples: 53670 53671 rclone backend restore oos:bucket/path/to/directory -o hours=HOURS 53672 rclone backend restore oos:bucket -o hours=HOURS 53673 \f[R] 53674 .fi 53675 .PP 53676 This flag also obeys the filters. 53677 Test first with --interactive/-i or --dry-run flags 53678 .IP 53679 .nf 53680 \f[C] 53681 rclone --interactive backend restore --include \[dq]*.txt\[dq] oos:bucket/path -o hours=72 53682 \f[R] 53683 .fi 53684 .PP 53685 All the objects shown will be marked for restore, then 53686 .IP 53687 .nf 53688 \f[C] 53689 rclone backend restore --include \[dq]*.txt\[dq] oos:bucket/path -o hours=72 53690 53691 It returns a list of status dictionaries with Object Name and Status 53692 keys. The Status will be \[dq]RESTORED\[dq]\[dq] if it was successful or an error message 53693 if not. 53694 53695 [ 53696 { 53697 \[dq]Object\[dq]: \[dq]test.txt\[dq] 53698 \[dq]Status\[dq]: \[dq]RESTORED\[dq], 53699 }, 53700 { 53701 \[dq]Object\[dq]: \[dq]test/file4.txt\[dq] 53702 \[dq]Status\[dq]: \[dq]RESTORED\[dq], 53703 } 53704 ] 53705 \f[R] 53706 .fi 53707 .PP 53708 Options: 53709 .IP \[bu] 2 53710 \[dq]hours\[dq]: The number of hours for which this object will be 53711 restored. 53712 Default is 24 hrs. 53713 .SS Tutorials 53714 .SS Mounting Buckets (https://rclone.org/oracleobjectstorage/tutorial_mount/) 53715 .SH QingStor 53716 .PP 53717 Paths are specified as \f[C]remote:bucket\f[R] (or \f[C]remote:\f[R] for 53718 the \f[C]lsd\f[R] command.) You may put subdirectories in too, e.g. 53719 \f[C]remote:bucket/path/to/dir\f[R]. 53720 .SS Configuration 53721 .PP 53722 Here is an example of making an QingStor configuration. 53723 First run 53724 .IP 53725 .nf 53726 \f[C] 53727 rclone config 53728 \f[R] 53729 .fi 53730 .PP 53731 This will guide you through an interactive setup process. 53732 .IP 53733 .nf 53734 \f[C] 53735 No remotes found, make a new one? 53736 n) New remote 53737 r) Rename remote 53738 c) Copy remote 53739 s) Set configuration password 53740 q) Quit config 53741 n/r/c/s/q> n 53742 name> remote 53743 Type of storage to configure. 53744 Choose a number from below, or type in your own value 53745 [snip] 53746 XX / QingStor Object Storage 53747 \[rs] \[dq]qingstor\[dq] 53748 [snip] 53749 Storage> qingstor 53750 Get QingStor credentials from runtime. Only applies if access_key_id and secret_access_key is blank. 53751 Choose a number from below, or type in your own value 53752 1 / Enter QingStor credentials in the next step 53753 \[rs] \[dq]false\[dq] 53754 2 / Get QingStor credentials from the environment (env vars or IAM) 53755 \[rs] \[dq]true\[dq] 53756 env_auth> 1 53757 QingStor Access Key ID - leave blank for anonymous access or runtime credentials. 53758 access_key_id> access_key 53759 QingStor Secret Access Key (password) - leave blank for anonymous access or runtime credentials. 53760 secret_access_key> secret_key 53761 Enter an endpoint URL to connection QingStor API. 53762 Leave blank will use the default value \[dq]https://qingstor.com:443\[dq] 53763 endpoint> 53764 Zone connect to. Default is \[dq]pek3a\[dq]. 53765 Choose a number from below, or type in your own value 53766 / The Beijing (China) Three Zone 53767 1 | Needs location constraint pek3a. 53768 \[rs] \[dq]pek3a\[dq] 53769 / The Shanghai (China) First Zone 53770 2 | Needs location constraint sh1a. 53771 \[rs] \[dq]sh1a\[dq] 53772 zone> 1 53773 Number of connection retry. 53774 Leave blank will use the default value \[dq]3\[dq]. 53775 connection_retries> 53776 Remote config 53777 -------------------- 53778 [remote] 53779 env_auth = false 53780 access_key_id = access_key 53781 secret_access_key = secret_key 53782 endpoint = 53783 zone = pek3a 53784 connection_retries = 53785 -------------------- 53786 y) Yes this is OK 53787 e) Edit this remote 53788 d) Delete this remote 53789 y/e/d> y 53790 \f[R] 53791 .fi 53792 .PP 53793 This remote is called \f[C]remote\f[R] and can now be used like this 53794 .PP 53795 See all buckets 53796 .IP 53797 .nf 53798 \f[C] 53799 rclone lsd remote: 53800 \f[R] 53801 .fi 53802 .PP 53803 Make a new bucket 53804 .IP 53805 .nf 53806 \f[C] 53807 rclone mkdir remote:bucket 53808 \f[R] 53809 .fi 53810 .PP 53811 List the contents of a bucket 53812 .IP 53813 .nf 53814 \f[C] 53815 rclone ls remote:bucket 53816 \f[R] 53817 .fi 53818 .PP 53819 Sync \f[C]/home/local/directory\f[R] to the remote bucket, deleting any 53820 excess files in the bucket. 53821 .IP 53822 .nf 53823 \f[C] 53824 rclone sync --interactive /home/local/directory remote:bucket 53825 \f[R] 53826 .fi 53827 .SS --fast-list 53828 .PP 53829 This remote supports \f[C]--fast-list\f[R] which allows you to use fewer 53830 transactions in exchange for more memory. 53831 See the rclone docs (https://rclone.org/docs/#fast-list) for more 53832 details. 53833 .SS Multipart uploads 53834 .PP 53835 rclone supports multipart uploads with QingStor which means that it can 53836 upload files bigger than 5 GiB. 53837 Note that files uploaded with multipart upload don\[aq]t have an MD5SUM. 53838 .PP 53839 Note that incomplete multipart uploads older than 24 hours can be 53840 removed with \f[C]rclone cleanup remote:bucket\f[R] just for one bucket 53841 \f[C]rclone cleanup remote:\f[R] for all buckets. 53842 QingStor does not ever remove incomplete multipart uploads so it may be 53843 necessary to run this from time to time. 53844 .SS Buckets and Zone 53845 .PP 53846 With QingStor you can list buckets (\f[C]rclone lsd\f[R]) using any 53847 zone, but you can only access the content of a bucket from the zone it 53848 was created in. 53849 If you attempt to access a bucket from the wrong zone, you will get an 53850 error, 53851 \f[C]incorrect zone, the bucket is not in \[aq]XXX\[aq] zone\f[R]. 53852 .SS Authentication 53853 .PP 53854 There are two ways to supply \f[C]rclone\f[R] with a set of QingStor 53855 credentials. 53856 In order of precedence: 53857 .IP \[bu] 2 53858 Directly in the rclone configuration file (as configured by 53859 \f[C]rclone config\f[R]) 53860 .RS 2 53861 .IP \[bu] 2 53862 set \f[C]access_key_id\f[R] and \f[C]secret_access_key\f[R] 53863 .RE 53864 .IP \[bu] 2 53865 Runtime configuration: 53866 .RS 2 53867 .IP \[bu] 2 53868 set \f[C]env_auth\f[R] to \f[C]true\f[R] in the config file 53869 .IP \[bu] 2 53870 Exporting the following environment variables before running 53871 \f[C]rclone\f[R] 53872 .RS 2 53873 .IP \[bu] 2 53874 Access Key ID: \f[C]QS_ACCESS_KEY_ID\f[R] or \f[C]QS_ACCESS_KEY\f[R] 53875 .IP \[bu] 2 53876 Secret Access Key: \f[C]QS_SECRET_ACCESS_KEY\f[R] or 53877 \f[C]QS_SECRET_KEY\f[R] 53878 .RE 53879 .RE 53880 .SS Restricted filename characters 53881 .PP 53882 The control characters 0x00-0x1F and / are replaced as in the default 53883 restricted characters 53884 set (https://rclone.org/overview/#restricted-characters). 53885 Note that 0x7F is not replaced. 53886 .PP 53887 Invalid UTF-8 bytes will also be 53888 replaced (https://rclone.org/overview/#invalid-utf8), as they can\[aq]t 53889 be used in JSON strings. 53890 .SS Standard options 53891 .PP 53892 Here are the Standard options specific to qingstor (QingCloud Object 53893 Storage). 53894 .SS --qingstor-env-auth 53895 .PP 53896 Get QingStor credentials from runtime. 53897 .PP 53898 Only applies if access_key_id and secret_access_key is blank. 53899 .PP 53900 Properties: 53901 .IP \[bu] 2 53902 Config: env_auth 53903 .IP \[bu] 2 53904 Env Var: RCLONE_QINGSTOR_ENV_AUTH 53905 .IP \[bu] 2 53906 Type: bool 53907 .IP \[bu] 2 53908 Default: false 53909 .IP \[bu] 2 53910 Examples: 53911 .RS 2 53912 .IP \[bu] 2 53913 \[dq]false\[dq] 53914 .RS 2 53915 .IP \[bu] 2 53916 Enter QingStor credentials in the next step. 53917 .RE 53918 .IP \[bu] 2 53919 \[dq]true\[dq] 53920 .RS 2 53921 .IP \[bu] 2 53922 Get QingStor credentials from the environment (env vars or IAM). 53923 .RE 53924 .RE 53925 .SS --qingstor-access-key-id 53926 .PP 53927 QingStor Access Key ID. 53928 .PP 53929 Leave blank for anonymous access or runtime credentials. 53930 .PP 53931 Properties: 53932 .IP \[bu] 2 53933 Config: access_key_id 53934 .IP \[bu] 2 53935 Env Var: RCLONE_QINGSTOR_ACCESS_KEY_ID 53936 .IP \[bu] 2 53937 Type: string 53938 .IP \[bu] 2 53939 Required: false 53940 .SS --qingstor-secret-access-key 53941 .PP 53942 QingStor Secret Access Key (password). 53943 .PP 53944 Leave blank for anonymous access or runtime credentials. 53945 .PP 53946 Properties: 53947 .IP \[bu] 2 53948 Config: secret_access_key 53949 .IP \[bu] 2 53950 Env Var: RCLONE_QINGSTOR_SECRET_ACCESS_KEY 53951 .IP \[bu] 2 53952 Type: string 53953 .IP \[bu] 2 53954 Required: false 53955 .SS --qingstor-endpoint 53956 .PP 53957 Enter an endpoint URL to connection QingStor API. 53958 .PP 53959 Leave blank will use the default value 53960 \[dq]https://qingstor.com:443\[dq]. 53961 .PP 53962 Properties: 53963 .IP \[bu] 2 53964 Config: endpoint 53965 .IP \[bu] 2 53966 Env Var: RCLONE_QINGSTOR_ENDPOINT 53967 .IP \[bu] 2 53968 Type: string 53969 .IP \[bu] 2 53970 Required: false 53971 .SS --qingstor-zone 53972 .PP 53973 Zone to connect to. 53974 .PP 53975 Default is \[dq]pek3a\[dq]. 53976 .PP 53977 Properties: 53978 .IP \[bu] 2 53979 Config: zone 53980 .IP \[bu] 2 53981 Env Var: RCLONE_QINGSTOR_ZONE 53982 .IP \[bu] 2 53983 Type: string 53984 .IP \[bu] 2 53985 Required: false 53986 .IP \[bu] 2 53987 Examples: 53988 .RS 2 53989 .IP \[bu] 2 53990 \[dq]pek3a\[dq] 53991 .RS 2 53992 .IP \[bu] 2 53993 The Beijing (China) Three Zone. 53994 .IP \[bu] 2 53995 Needs location constraint pek3a. 53996 .RE 53997 .IP \[bu] 2 53998 \[dq]sh1a\[dq] 53999 .RS 2 54000 .IP \[bu] 2 54001 The Shanghai (China) First Zone. 54002 .IP \[bu] 2 54003 Needs location constraint sh1a. 54004 .RE 54005 .IP \[bu] 2 54006 \[dq]gd2a\[dq] 54007 .RS 2 54008 .IP \[bu] 2 54009 The Guangdong (China) Second Zone. 54010 .IP \[bu] 2 54011 Needs location constraint gd2a. 54012 .RE 54013 .RE 54014 .SS Advanced options 54015 .PP 54016 Here are the Advanced options specific to qingstor (QingCloud Object 54017 Storage). 54018 .SS --qingstor-connection-retries 54019 .PP 54020 Number of connection retries. 54021 .PP 54022 Properties: 54023 .IP \[bu] 2 54024 Config: connection_retries 54025 .IP \[bu] 2 54026 Env Var: RCLONE_QINGSTOR_CONNECTION_RETRIES 54027 .IP \[bu] 2 54028 Type: int 54029 .IP \[bu] 2 54030 Default: 3 54031 .SS --qingstor-upload-cutoff 54032 .PP 54033 Cutoff for switching to chunked upload. 54034 .PP 54035 Any files larger than this will be uploaded in chunks of chunk_size. 54036 The minimum is 0 and the maximum is 5 GiB. 54037 .PP 54038 Properties: 54039 .IP \[bu] 2 54040 Config: upload_cutoff 54041 .IP \[bu] 2 54042 Env Var: RCLONE_QINGSTOR_UPLOAD_CUTOFF 54043 .IP \[bu] 2 54044 Type: SizeSuffix 54045 .IP \[bu] 2 54046 Default: 200Mi 54047 .SS --qingstor-chunk-size 54048 .PP 54049 Chunk size to use for uploading. 54050 .PP 54051 When uploading files larger than upload_cutoff they will be uploaded as 54052 multipart uploads using this chunk size. 54053 .PP 54054 Note that \[dq]--qingstor-upload-concurrency\[dq] chunks of this size 54055 are buffered in memory per transfer. 54056 .PP 54057 If you are transferring large files over high-speed links and you have 54058 enough memory, then increasing this will speed up the transfers. 54059 .PP 54060 Properties: 54061 .IP \[bu] 2 54062 Config: chunk_size 54063 .IP \[bu] 2 54064 Env Var: RCLONE_QINGSTOR_CHUNK_SIZE 54065 .IP \[bu] 2 54066 Type: SizeSuffix 54067 .IP \[bu] 2 54068 Default: 4Mi 54069 .SS --qingstor-upload-concurrency 54070 .PP 54071 Concurrency for multipart uploads. 54072 .PP 54073 This is the number of chunks of the same file that are uploaded 54074 concurrently. 54075 .PP 54076 NB if you set this to > 1 then the checksums of multipart uploads become 54077 corrupted (the uploads themselves are not corrupted though). 54078 .PP 54079 If you are uploading small numbers of large files over high-speed links 54080 and these uploads do not fully utilize your bandwidth, then increasing 54081 this may help to speed up the transfers. 54082 .PP 54083 Properties: 54084 .IP \[bu] 2 54085 Config: upload_concurrency 54086 .IP \[bu] 2 54087 Env Var: RCLONE_QINGSTOR_UPLOAD_CONCURRENCY 54088 .IP \[bu] 2 54089 Type: int 54090 .IP \[bu] 2 54091 Default: 1 54092 .SS --qingstor-encoding 54093 .PP 54094 The encoding for the backend. 54095 .PP 54096 See the encoding section in the 54097 overview (https://rclone.org/overview/#encoding) for more info. 54098 .PP 54099 Properties: 54100 .IP \[bu] 2 54101 Config: encoding 54102 .IP \[bu] 2 54103 Env Var: RCLONE_QINGSTOR_ENCODING 54104 .IP \[bu] 2 54105 Type: Encoding 54106 .IP \[bu] 2 54107 Default: Slash,Ctl,InvalidUtf8 54108 .SS --qingstor-description 54109 .PP 54110 Description of the remote 54111 .PP 54112 Properties: 54113 .IP \[bu] 2 54114 Config: description 54115 .IP \[bu] 2 54116 Env Var: RCLONE_QINGSTOR_DESCRIPTION 54117 .IP \[bu] 2 54118 Type: string 54119 .IP \[bu] 2 54120 Required: false 54121 .SS Limitations 54122 .PP 54123 \f[C]rclone about\f[R] is not supported by the qingstor backend. 54124 Backends without this capability cannot determine free space for an 54125 rclone mount or use policy \f[C]mfs\f[R] (most free space) as a member 54126 of an rclone union remote. 54127 .PP 54128 See List of backends that do not support rclone 54129 about (https://rclone.org/overview/#optional-features) and rclone 54130 about (https://rclone.org/commands/rclone_about/) 54131 .SH Quatrix 54132 .PP 54133 Quatrix by Maytech is Quatrix Secure Compliant File Sharing | 54134 Maytech (https://www.maytech.net/products/quatrix-business). 54135 .PP 54136 Paths are specified as \f[C]remote:path\f[R] 54137 .PP 54138 Paths may be as deep as required, e.g., 54139 \f[C]remote:directory/subdirectory\f[R]. 54140 .PP 54141 The initial setup for Quatrix involves getting an API Key from Quatrix. 54142 You can get the API key in the user\[aq]s profile at 54143 \f[C]https://<account>/profile/api-keys\f[R] or with the help of the API 54144 - 54145 https://docs.maytech.net/quatrix/quatrix-api/api-explorer#/API-Key/post_api_key_create. 54146 .PP 54147 See complete Swagger documentation for Quatrix - 54148 https://docs.maytech.net/quatrix/quatrix-api/api-explorer 54149 .SS Configuration 54150 .PP 54151 Here is an example of how to make a remote called \f[C]remote\f[R]. 54152 First run: 54153 .IP 54154 .nf 54155 \f[C] 54156 rclone config 54157 \f[R] 54158 .fi 54159 .PP 54160 This will guide you through an interactive setup process: 54161 .IP 54162 .nf 54163 \f[C] 54164 No remotes found, make a new one? 54165 n) New remote 54166 s) Set configuration password 54167 q) Quit config 54168 n/s/q> n 54169 name> remote 54170 Type of storage to configure. 54171 Choose a number from below, or type in your own value 54172 [snip] 54173 XX / Quatrix by Maytech 54174 \[rs] \[dq]quatrix\[dq] 54175 [snip] 54176 Storage> quatrix 54177 API key for accessing Quatrix account. 54178 api_key> your_api_key 54179 Host name of Quatrix account. 54180 host> example.quatrix.it 54181 54182 -------------------- 54183 [remote] 54184 api_key = your_api_key 54185 host = example.quatrix.it 54186 -------------------- 54187 y) Yes this is OK 54188 e) Edit this remote 54189 d) Delete this remote 54190 y/e/d> y 54191 \f[R] 54192 .fi 54193 .PP 54194 Once configured you can then use \f[C]rclone\f[R] like this, 54195 .PP 54196 List directories in top level of your Quatrix 54197 .IP 54198 .nf 54199 \f[C] 54200 rclone lsd remote: 54201 \f[R] 54202 .fi 54203 .PP 54204 List all the files in your Quatrix 54205 .IP 54206 .nf 54207 \f[C] 54208 rclone ls remote: 54209 \f[R] 54210 .fi 54211 .PP 54212 To copy a local directory to an Quatrix directory called backup 54213 .IP 54214 .nf 54215 \f[C] 54216 rclone copy /home/source remote:backup 54217 \f[R] 54218 .fi 54219 .SS API key validity 54220 .PP 54221 API Key is created with no expiration date. 54222 It will be valid until you delete or deactivate it in your account. 54223 After disabling, the API Key can be enabled back. 54224 If the API Key was deleted and a new key was created, you can update it 54225 in rclone config. 54226 The same happens if the hostname was changed. 54227 .IP 54228 .nf 54229 \f[C] 54230 $ rclone config 54231 Current remotes: 54232 54233 Name Type 54234 ==== ==== 54235 remote quatrix 54236 54237 e) Edit existing remote 54238 n) New remote 54239 d) Delete remote 54240 r) Rename remote 54241 c) Copy remote 54242 s) Set configuration password 54243 q) Quit config 54244 e/n/d/r/c/s/q> e 54245 Choose a number from below, or type in an existing value 54246 1 > remote 54247 remote> remote 54248 -------------------- 54249 [remote] 54250 type = quatrix 54251 host = some_host.quatrix.it 54252 api_key = your_api_key 54253 -------------------- 54254 Edit remote 54255 Option api_key. 54256 API key for accessing Quatrix account 54257 Enter a string value. Press Enter for the default (your_api_key) 54258 api_key> 54259 Option host. 54260 Host name of Quatrix account 54261 Enter a string value. Press Enter for the default (some_host.quatrix.it). 54262 54263 -------------------- 54264 [remote] 54265 type = quatrix 54266 host = some_host.quatrix.it 54267 api_key = your_api_key 54268 -------------------- 54269 y) Yes this is OK 54270 e) Edit this remote 54271 d) Delete this remote 54272 y/e/d> y 54273 \f[R] 54274 .fi 54275 .SS Modification times and hashes 54276 .PP 54277 Quatrix allows modification times to be set on objects accurate to 1 54278 microsecond. 54279 These will be used to detect whether objects need syncing or not. 54280 .PP 54281 Quatrix does not support hashes, so you cannot use the 54282 \f[C]--checksum\f[R] flag. 54283 .SS Restricted filename characters 54284 .PP 54285 File names in Quatrix are case sensitive and have limitations like the 54286 maximum length of a filename is 255, and the minimum length is 1. 54287 A file name cannot be equal to \f[C].\f[R] or \f[C]..\f[R] nor contain 54288 \f[C]/\f[R] , \f[C]\[rs]\f[R] or non-printable ascii. 54289 .SS Transfers 54290 .PP 54291 For files above 50 MiB rclone will use a chunked transfer. 54292 Rclone will upload up to \f[C]--transfers\f[R] chunks at the same time 54293 (shared among all multipart uploads). 54294 Chunks are buffered in memory, and the minimal chunk size is 10_000_000 54295 bytes by default, and it can be changed in the advanced configuration, 54296 so increasing \f[C]--transfers\f[R] will increase the memory use. 54297 The chunk size has a maximum size limit, which is set to 100_000_000 54298 bytes by default and can be changed in the advanced configuration. 54299 The size of the uploaded chunk will dynamically change depending on the 54300 upload speed. 54301 The total memory use equals the number of transfers multiplied by the 54302 minimal chunk size. 54303 In case there\[aq]s free memory allocated for the upload (which equals 54304 the difference of \f[C]maximal_summary_chunk_size\f[R] and 54305 \f[C]minimal_chunk_size\f[R] * \f[C]transfers\f[R]), the chunk size may 54306 increase in case of high upload speed. 54307 As well as it can decrease in case of upload speed problems. 54308 If no free memory is available, all chunks will equal 54309 \f[C]minimal_chunk_size\f[R]. 54310 .SS Deleting files 54311 .PP 54312 Files you delete with rclone will end up in Trash and be stored there 54313 for 30 days. 54314 Quatrix also provides an API to permanently delete files and an API to 54315 empty the Trash so that you can remove files permanently from your 54316 account. 54317 .SS Standard options 54318 .PP 54319 Here are the Standard options specific to quatrix (Quatrix by Maytech). 54320 .SS --quatrix-api-key 54321 .PP 54322 API key for accessing Quatrix account 54323 .PP 54324 Properties: 54325 .IP \[bu] 2 54326 Config: api_key 54327 .IP \[bu] 2 54328 Env Var: RCLONE_QUATRIX_API_KEY 54329 .IP \[bu] 2 54330 Type: string 54331 .IP \[bu] 2 54332 Required: true 54333 .SS --quatrix-host 54334 .PP 54335 Host name of Quatrix account 54336 .PP 54337 Properties: 54338 .IP \[bu] 2 54339 Config: host 54340 .IP \[bu] 2 54341 Env Var: RCLONE_QUATRIX_HOST 54342 .IP \[bu] 2 54343 Type: string 54344 .IP \[bu] 2 54345 Required: true 54346 .SS Advanced options 54347 .PP 54348 Here are the Advanced options specific to quatrix (Quatrix by Maytech). 54349 .SS --quatrix-encoding 54350 .PP 54351 The encoding for the backend. 54352 .PP 54353 See the encoding section in the 54354 overview (https://rclone.org/overview/#encoding) for more info. 54355 .PP 54356 Properties: 54357 .IP \[bu] 2 54358 Config: encoding 54359 .IP \[bu] 2 54360 Env Var: RCLONE_QUATRIX_ENCODING 54361 .IP \[bu] 2 54362 Type: Encoding 54363 .IP \[bu] 2 54364 Default: Slash,BackSlash,Del,Ctl,InvalidUtf8,Dot 54365 .SS --quatrix-effective-upload-time 54366 .PP 54367 Wanted upload time for one chunk 54368 .PP 54369 Properties: 54370 .IP \[bu] 2 54371 Config: effective_upload_time 54372 .IP \[bu] 2 54373 Env Var: RCLONE_QUATRIX_EFFECTIVE_UPLOAD_TIME 54374 .IP \[bu] 2 54375 Type: string 54376 .IP \[bu] 2 54377 Default: \[dq]4s\[dq] 54378 .SS --quatrix-minimal-chunk-size 54379 .PP 54380 The minimal size for one chunk 54381 .PP 54382 Properties: 54383 .IP \[bu] 2 54384 Config: minimal_chunk_size 54385 .IP \[bu] 2 54386 Env Var: RCLONE_QUATRIX_MINIMAL_CHUNK_SIZE 54387 .IP \[bu] 2 54388 Type: SizeSuffix 54389 .IP \[bu] 2 54390 Default: 9.537Mi 54391 .SS --quatrix-maximal-summary-chunk-size 54392 .PP 54393 The maximal summary for all chunks. 54394 It should not be less than 54395 \[aq]transfers\[aq]*\[aq]minimal_chunk_size\[aq] 54396 .PP 54397 Properties: 54398 .IP \[bu] 2 54399 Config: maximal_summary_chunk_size 54400 .IP \[bu] 2 54401 Env Var: RCLONE_QUATRIX_MAXIMAL_SUMMARY_CHUNK_SIZE 54402 .IP \[bu] 2 54403 Type: SizeSuffix 54404 .IP \[bu] 2 54405 Default: 95.367Mi 54406 .SS --quatrix-hard-delete 54407 .PP 54408 Delete files permanently rather than putting them into the trash 54409 .PP 54410 Properties: 54411 .IP \[bu] 2 54412 Config: hard_delete 54413 .IP \[bu] 2 54414 Env Var: RCLONE_QUATRIX_HARD_DELETE 54415 .IP \[bu] 2 54416 Type: bool 54417 .IP \[bu] 2 54418 Default: false 54419 .SS --quatrix-skip-project-folders 54420 .PP 54421 Skip project folders in operations 54422 .PP 54423 Properties: 54424 .IP \[bu] 2 54425 Config: skip_project_folders 54426 .IP \[bu] 2 54427 Env Var: RCLONE_QUATRIX_SKIP_PROJECT_FOLDERS 54428 .IP \[bu] 2 54429 Type: bool 54430 .IP \[bu] 2 54431 Default: false 54432 .SS --quatrix-description 54433 .PP 54434 Description of the remote 54435 .PP 54436 Properties: 54437 .IP \[bu] 2 54438 Config: description 54439 .IP \[bu] 2 54440 Env Var: RCLONE_QUATRIX_DESCRIPTION 54441 .IP \[bu] 2 54442 Type: string 54443 .IP \[bu] 2 54444 Required: false 54445 .SS Storage usage 54446 .PP 54447 The storage usage in Quatrix is restricted to the account during the 54448 purchase. 54449 You can restrict any user with a smaller storage limit. 54450 The account limit is applied if the user has no custom storage limit. 54451 Once you\[aq]ve reached the limit, the upload of files will fail. 54452 This can be fixed by freeing up the space or increasing the quota. 54453 .SS Server-side operations 54454 .PP 54455 Quatrix supports server-side operations (copy and move). 54456 In case of conflict, files are overwritten during server-side operation. 54457 .SH Sia 54458 .PP 54459 Sia (sia.tech (https://sia.tech/)) is a decentralized cloud storage 54460 platform based on the blockchain (https://wikipedia.org/wiki/Blockchain) 54461 technology. 54462 With rclone you can use it like any other remote filesystem or mount Sia 54463 folders locally. 54464 The technology behind it involves a number of new concepts such as 54465 Siacoins and Wallet, Blockchain and Consensus, Renting and Hosting, and 54466 so on. 54467 If you are new to it, you\[aq]d better first familiarize yourself using 54468 their excellent support documentation (https://support.sia.tech/). 54469 .SS Introduction 54470 .PP 54471 Before you can use rclone with Sia, you will need to have a running copy 54472 of \f[C]Sia-UI\f[R] or \f[C]siad\f[R] (the Sia daemon) locally on your 54473 computer or on local network (e.g. 54474 a NAS). 54475 Please follow the Get started (https://sia.tech/get-started) guide and 54476 install one. 54477 .PP 54478 rclone interacts with Sia network by talking to the Sia daemon via HTTP 54479 API (https://sia.tech/docs/) which is usually available on port 54480 \f[I]9980\f[R]. 54481 By default you will run the daemon locally on the same computer so 54482 it\[aq]s safe to leave the API password blank (the API URL will be 54483 \f[C]http://127.0.0.1:9980\f[R] making external access impossible). 54484 .PP 54485 However, if you want to access Sia daemon running on another node, for 54486 example due to memory constraints or because you want to share single 54487 daemon between several rclone and Sia-UI instances, you\[aq]ll need to 54488 make a few more provisions: - Ensure you have \f[I]Sia daemon\f[R] 54489 installed directly or in a docker 54490 container (https://github.com/SiaFoundation/siad/pkgs/container/siad) 54491 because Sia-UI does not support this mode natively. 54492 - Run it on externally accessible port, for example provide 54493 \f[C]--api-addr :9980\f[R] and \f[C]--disable-api-security\f[R] 54494 arguments on the daemon command line. 54495 - Enforce API password for the \f[C]siad\f[R] daemon via environment 54496 variable \f[C]SIA_API_PASSWORD\f[R] or text file named 54497 \f[C]apipassword\f[R] in the daemon directory. 54498 - Set rclone backend option \f[C]api_password\f[R] taking it from above 54499 locations. 54500 .PP 54501 Notes: 1. 54502 If your wallet is locked, rclone cannot unlock it automatically. 54503 You should either unlock it in advance by using Sia-UI or via command 54504 line \f[C]siac wallet unlock\f[R]. 54505 Alternatively you can make \f[C]siad\f[R] unlock your wallet 54506 automatically upon startup by running it with environment variable 54507 \f[C]SIA_WALLET_PASSWORD\f[R]. 54508 2. 54509 If \f[C]siad\f[R] cannot find the \f[C]SIA_API_PASSWORD\f[R] variable or 54510 the \f[C]apipassword\f[R] file in the \f[C]SIA_DIR\f[R] directory, it 54511 will generate a random password and store in the text file named 54512 \f[C]apipassword\f[R] under \f[C]YOUR_HOME/.sia/\f[R] directory on Unix 54513 or 54514 \f[C]C:\[rs]Users\[rs]YOUR_HOME\[rs]AppData\[rs]Local\[rs]Sia\[rs]apipassword\f[R] 54515 on Windows. 54516 Remember this when you configure password in rclone. 54517 3. 54518 The only way to use \f[C]siad\f[R] without API password is to run it 54519 \f[B]on localhost\f[R] with command line argument 54520 \f[C]--authorize-api=false\f[R], but this is insecure and \f[B]strongly 54521 discouraged\f[R]. 54522 .SS Configuration 54523 .PP 54524 Here is an example of how to make a \f[C]sia\f[R] remote called 54525 \f[C]mySia\f[R]. 54526 First, run: 54527 .IP 54528 .nf 54529 \f[C] 54530 rclone config 54531 \f[R] 54532 .fi 54533 .PP 54534 This will guide you through an interactive setup process: 54535 .IP 54536 .nf 54537 \f[C] 54538 No remotes found, make a new one? 54539 n) New remote 54540 s) Set configuration password 54541 q) Quit config 54542 n/s/q> n 54543 name> mySia 54544 Type of storage to configure. 54545 Enter a string value. Press Enter for the default (\[dq]\[dq]). 54546 Choose a number from below, or type in your own value 54547 \&... 54548 29 / Sia Decentralized Cloud 54549 \[rs] \[dq]sia\[dq] 54550 \&... 54551 Storage> sia 54552 Sia daemon API URL, like http://sia.daemon.host:9980. 54553 Note that siad must run with --disable-api-security to open API port for other hosts (not recommended). 54554 Keep default if Sia daemon runs on localhost. 54555 Enter a string value. Press Enter for the default (\[dq]http://127.0.0.1:9980\[dq]). 54556 api_url> http://127.0.0.1:9980 54557 Sia Daemon API Password. 54558 Can be found in the apipassword file located in HOME/.sia/ or in the daemon directory. 54559 y) Yes type in my own password 54560 g) Generate random password 54561 n) No leave this optional password blank (default) 54562 y/g/n> y 54563 Enter the password: 54564 password: 54565 Confirm the password: 54566 password: 54567 Edit advanced config? 54568 y) Yes 54569 n) No (default) 54570 y/n> n 54571 -------------------- 54572 [mySia] 54573 type = sia 54574 api_url = http://127.0.0.1:9980 54575 api_password = *** ENCRYPTED *** 54576 -------------------- 54577 y) Yes this is OK (default) 54578 e) Edit this remote 54579 d) Delete this remote 54580 y/e/d> y 54581 \f[R] 54582 .fi 54583 .PP 54584 Once configured, you can then use \f[C]rclone\f[R] like this: 54585 .IP \[bu] 2 54586 List directories in top level of your Sia storage 54587 .IP 54588 .nf 54589 \f[C] 54590 rclone lsd mySia: 54591 \f[R] 54592 .fi 54593 .IP \[bu] 2 54594 List all the files in your Sia storage 54595 .IP 54596 .nf 54597 \f[C] 54598 rclone ls mySia: 54599 \f[R] 54600 .fi 54601 .IP \[bu] 2 54602 Upload a local directory to the Sia directory called \f[I]backup\f[R] 54603 .IP 54604 .nf 54605 \f[C] 54606 rclone copy /home/source mySia:backup 54607 \f[R] 54608 .fi 54609 .SS Standard options 54610 .PP 54611 Here are the Standard options specific to sia (Sia Decentralized Cloud). 54612 .SS --sia-api-url 54613 .PP 54614 Sia daemon API URL, like http://sia.daemon.host:9980. 54615 .PP 54616 Note that siad must run with --disable-api-security to open API port for 54617 other hosts (not recommended). 54618 Keep default if Sia daemon runs on localhost. 54619 .PP 54620 Properties: 54621 .IP \[bu] 2 54622 Config: api_url 54623 .IP \[bu] 2 54624 Env Var: RCLONE_SIA_API_URL 54625 .IP \[bu] 2 54626 Type: string 54627 .IP \[bu] 2 54628 Default: \[dq]http://127.0.0.1:9980\[dq] 54629 .SS --sia-api-password 54630 .PP 54631 Sia Daemon API Password. 54632 .PP 54633 Can be found in the apipassword file located in HOME/.sia/ or in the 54634 daemon directory. 54635 .PP 54636 \f[B]NB\f[R] Input to this must be obscured - see rclone 54637 obscure (https://rclone.org/commands/rclone_obscure/). 54638 .PP 54639 Properties: 54640 .IP \[bu] 2 54641 Config: api_password 54642 .IP \[bu] 2 54643 Env Var: RCLONE_SIA_API_PASSWORD 54644 .IP \[bu] 2 54645 Type: string 54646 .IP \[bu] 2 54647 Required: false 54648 .SS Advanced options 54649 .PP 54650 Here are the Advanced options specific to sia (Sia Decentralized Cloud). 54651 .SS --sia-user-agent 54652 .PP 54653 Siad User Agent 54654 .PP 54655 Sia daemon requires the \[aq]Sia-Agent\[aq] user agent by default for 54656 security 54657 .PP 54658 Properties: 54659 .IP \[bu] 2 54660 Config: user_agent 54661 .IP \[bu] 2 54662 Env Var: RCLONE_SIA_USER_AGENT 54663 .IP \[bu] 2 54664 Type: string 54665 .IP \[bu] 2 54666 Default: \[dq]Sia-Agent\[dq] 54667 .SS --sia-encoding 54668 .PP 54669 The encoding for the backend. 54670 .PP 54671 See the encoding section in the 54672 overview (https://rclone.org/overview/#encoding) for more info. 54673 .PP 54674 Properties: 54675 .IP \[bu] 2 54676 Config: encoding 54677 .IP \[bu] 2 54678 Env Var: RCLONE_SIA_ENCODING 54679 .IP \[bu] 2 54680 Type: Encoding 54681 .IP \[bu] 2 54682 Default: Slash,Question,Hash,Percent,Del,Ctl,InvalidUtf8,Dot 54683 .SS --sia-description 54684 .PP 54685 Description of the remote 54686 .PP 54687 Properties: 54688 .IP \[bu] 2 54689 Config: description 54690 .IP \[bu] 2 54691 Env Var: RCLONE_SIA_DESCRIPTION 54692 .IP \[bu] 2 54693 Type: string 54694 .IP \[bu] 2 54695 Required: false 54696 .SS Limitations 54697 .IP \[bu] 2 54698 Modification times not supported 54699 .IP \[bu] 2 54700 Checksums not supported 54701 .IP \[bu] 2 54702 \f[C]rclone about\f[R] not supported 54703 .IP \[bu] 2 54704 rclone can work only with \f[I]Siad\f[R] or \f[I]Sia-UI\f[R] at the 54705 moment, the \f[B]SkyNet daemon is not supported yet.\f[R] 54706 .IP \[bu] 2 54707 Sia does not allow control characters or symbols like question and pound 54708 signs in file names. 54709 rclone will transparently encode (https://rclone.org/overview/#encoding) 54710 them for you, but you\[aq]d better be aware 54711 .SH Swift 54712 .PP 54713 Swift refers to OpenStack Object 54714 Storage (https://docs.openstack.org/swift/latest/). 54715 Commercial implementations of that being: 54716 .IP \[bu] 2 54717 Rackspace Cloud Files (https://www.rackspace.com/cloud/files/) 54718 .IP \[bu] 2 54719 Memset Memstore (https://www.memset.com/cloud/storage/) 54720 .IP \[bu] 2 54721 OVH Object 54722 Storage (https://www.ovh.co.uk/public-cloud/storage/object-storage/) 54723 .IP \[bu] 2 54724 Oracle Cloud 54725 Storage (https://docs.oracle.com/en-us/iaas/integration/doc/configure-object-storage.html) 54726 .IP \[bu] 2 54727 Blomp Cloud Storage (https://www.blomp.com/cloud-storage/) 54728 .IP \[bu] 2 54729 IBM Bluemix Cloud ObjectStorage 54730 Swift (https://console.bluemix.net/docs/infrastructure/objectstorage-swift/index.html) 54731 .PP 54732 Paths are specified as \f[C]remote:container\f[R] (or \f[C]remote:\f[R] 54733 for the \f[C]lsd\f[R] command.) You may put subdirectories in too, e.g. 54734 \f[C]remote:container/path/to/dir\f[R]. 54735 .SS Configuration 54736 .PP 54737 Here is an example of making a swift configuration. 54738 First run 54739 .IP 54740 .nf 54741 \f[C] 54742 rclone config 54743 \f[R] 54744 .fi 54745 .PP 54746 This will guide you through an interactive setup process. 54747 .IP 54748 .nf 54749 \f[C] 54750 No remotes found, make a new one? 54751 n) New remote 54752 s) Set configuration password 54753 q) Quit config 54754 n/s/q> n 54755 name> remote 54756 Type of storage to configure. 54757 Choose a number from below, or type in your own value 54758 [snip] 54759 XX / OpenStack Swift (Rackspace Cloud Files, Blomp Cloud Storage, Memset Memstore, OVH) 54760 \[rs] \[dq]swift\[dq] 54761 [snip] 54762 Storage> swift 54763 Get swift credentials from environment variables in standard OpenStack form. 54764 Choose a number from below, or type in your own value 54765 1 / Enter swift credentials in the next step 54766 \[rs] \[dq]false\[dq] 54767 2 / Get swift credentials from environment vars. Leave other fields blank if using this. 54768 \[rs] \[dq]true\[dq] 54769 env_auth> true 54770 User name to log in (OS_USERNAME). 54771 user> 54772 API key or password (OS_PASSWORD). 54773 key> 54774 Authentication URL for server (OS_AUTH_URL). 54775 Choose a number from below, or type in your own value 54776 1 / Rackspace US 54777 \[rs] \[dq]https://auth.api.rackspacecloud.com/v1.0\[dq] 54778 2 / Rackspace UK 54779 \[rs] \[dq]https://lon.auth.api.rackspacecloud.com/v1.0\[dq] 54780 3 / Rackspace v2 54781 \[rs] \[dq]https://identity.api.rackspacecloud.com/v2.0\[dq] 54782 4 / Memset Memstore UK 54783 \[rs] \[dq]https://auth.storage.memset.com/v1.0\[dq] 54784 5 / Memset Memstore UK v2 54785 \[rs] \[dq]https://auth.storage.memset.com/v2.0\[dq] 54786 6 / OVH 54787 \[rs] \[dq]https://auth.cloud.ovh.net/v3\[dq] 54788 7 / Blomp Cloud Storage 54789 \[rs] \[dq]https://authenticate.ain.net\[dq] 54790 auth> 54791 User ID to log in - optional - most swift systems use user and leave this blank (v3 auth) (OS_USER_ID). 54792 user_id> 54793 User domain - optional (v3 auth) (OS_USER_DOMAIN_NAME) 54794 domain> 54795 Tenant name - optional for v1 auth, this or tenant_id required otherwise (OS_TENANT_NAME or OS_PROJECT_NAME) 54796 tenant> 54797 Tenant ID - optional for v1 auth, this or tenant required otherwise (OS_TENANT_ID) 54798 tenant_id> 54799 Tenant domain - optional (v3 auth) (OS_PROJECT_DOMAIN_NAME) 54800 tenant_domain> 54801 Region name - optional (OS_REGION_NAME) 54802 region> 54803 Storage URL - optional (OS_STORAGE_URL) 54804 storage_url> 54805 Auth Token from alternate authentication - optional (OS_AUTH_TOKEN) 54806 auth_token> 54807 AuthVersion - optional - set to (1,2,3) if your auth URL has no version (ST_AUTH_VERSION) 54808 auth_version> 54809 Endpoint type to choose from the service catalogue (OS_ENDPOINT_TYPE) 54810 Choose a number from below, or type in your own value 54811 1 / Public (default, choose this if not sure) 54812 \[rs] \[dq]public\[dq] 54813 2 / Internal (use internal service net) 54814 \[rs] \[dq]internal\[dq] 54815 3 / Admin 54816 \[rs] \[dq]admin\[dq] 54817 endpoint_type> 54818 Remote config 54819 -------------------- 54820 [test] 54821 env_auth = true 54822 user = 54823 key = 54824 auth = 54825 user_id = 54826 domain = 54827 tenant = 54828 tenant_id = 54829 tenant_domain = 54830 region = 54831 storage_url = 54832 auth_token = 54833 auth_version = 54834 endpoint_type = 54835 -------------------- 54836 y) Yes this is OK 54837 e) Edit this remote 54838 d) Delete this remote 54839 y/e/d> y 54840 \f[R] 54841 .fi 54842 .PP 54843 This remote is called \f[C]remote\f[R] and can now be used like this 54844 .PP 54845 See all containers 54846 .IP 54847 .nf 54848 \f[C] 54849 rclone lsd remote: 54850 \f[R] 54851 .fi 54852 .PP 54853 Make a new container 54854 .IP 54855 .nf 54856 \f[C] 54857 rclone mkdir remote:container 54858 \f[R] 54859 .fi 54860 .PP 54861 List the contents of a container 54862 .IP 54863 .nf 54864 \f[C] 54865 rclone ls remote:container 54866 \f[R] 54867 .fi 54868 .PP 54869 Sync \f[C]/home/local/directory\f[R] to the remote container, deleting 54870 any excess files in the container. 54871 .IP 54872 .nf 54873 \f[C] 54874 rclone sync --interactive /home/local/directory remote:container 54875 \f[R] 54876 .fi 54877 .SS Configuration from an OpenStack credentials file 54878 .PP 54879 An OpenStack credentials file typically looks something something like 54880 this (without the comments) 54881 .IP 54882 .nf 54883 \f[C] 54884 export OS_AUTH_URL=https://a.provider.net/v2.0 54885 export OS_TENANT_ID=ffffffffffffffffffffffffffffffff 54886 export OS_TENANT_NAME=\[dq]1234567890123456\[dq] 54887 export OS_USERNAME=\[dq]123abc567xy\[dq] 54888 echo \[dq]Please enter your OpenStack Password: \[dq] 54889 read -sr OS_PASSWORD_INPUT 54890 export OS_PASSWORD=$OS_PASSWORD_INPUT 54891 export OS_REGION_NAME=\[dq]SBG1\[dq] 54892 if [ -z \[dq]$OS_REGION_NAME\[dq] ]; then unset OS_REGION_NAME; fi 54893 \f[R] 54894 .fi 54895 .PP 54896 The config file needs to look something like this where 54897 \f[C]$OS_USERNAME\f[R] represents the value of the \f[C]OS_USERNAME\f[R] 54898 variable - \f[C]123abc567xy\f[R] in the example above. 54899 .IP 54900 .nf 54901 \f[C] 54902 [remote] 54903 type = swift 54904 user = $OS_USERNAME 54905 key = $OS_PASSWORD 54906 auth = $OS_AUTH_URL 54907 tenant = $OS_TENANT_NAME 54908 \f[R] 54909 .fi 54910 .PP 54911 Note that you may (or may not) need to set \f[C]region\f[R] too - try 54912 without first. 54913 .SS Configuration from the environment 54914 .PP 54915 If you prefer you can configure rclone to use swift using a standard set 54916 of OpenStack environment variables. 54917 .PP 54918 When you run through the config, make sure you choose \f[C]true\f[R] for 54919 \f[C]env_auth\f[R] and leave everything else blank. 54920 .PP 54921 rclone will then set any empty config parameters from the environment 54922 using standard OpenStack environment variables. 54923 There is a list of the 54924 variables (https://godoc.org/github.com/ncw/swift#Connection.ApplyEnvironment) 54925 in the docs for the swift library. 54926 .SS Using an alternate authentication method 54927 .PP 54928 If your OpenStack installation uses a non-standard authentication method 54929 that might not be yet supported by rclone or the underlying swift 54930 library, you can authenticate externally (e.g. 54931 calling manually the \f[C]openstack\f[R] commands to get a token). 54932 Then, you just need to pass the two configuration variables 54933 \f[C]auth_token\f[R] and \f[C]storage_url\f[R]. 54934 If they are both provided, the other variables are ignored. 54935 rclone will not try to authenticate but instead assume it is already 54936 authenticated and use these two variables to access the OpenStack 54937 installation. 54938 .SS Using rclone without a config file 54939 .PP 54940 You can use rclone with swift without a config file, if desired, like 54941 this: 54942 .IP 54943 .nf 54944 \f[C] 54945 source openstack-credentials-file 54946 export RCLONE_CONFIG_MYREMOTE_TYPE=swift 54947 export RCLONE_CONFIG_MYREMOTE_ENV_AUTH=true 54948 rclone lsd myremote: 54949 \f[R] 54950 .fi 54951 .SS --fast-list 54952 .PP 54953 This remote supports \f[C]--fast-list\f[R] which allows you to use fewer 54954 transactions in exchange for more memory. 54955 See the rclone docs (https://rclone.org/docs/#fast-list) for more 54956 details. 54957 .SS --update and --use-server-modtime 54958 .PP 54959 As noted below, the modified time is stored on metadata on the object. 54960 It is used by default for all operations that require checking the time 54961 a file was last updated. 54962 It allows rclone to treat the remote more like a true filesystem, but it 54963 is inefficient because it requires an extra API call to retrieve the 54964 metadata. 54965 .PP 54966 For many operations, the time the object was last uploaded to the remote 54967 is sufficient to determine if it is \[dq]dirty\[dq]. 54968 By using \f[C]--update\f[R] along with \f[C]--use-server-modtime\f[R], 54969 you can avoid the extra API call and simply upload files whose local 54970 modtime is newer than the time it was last uploaded. 54971 .SS Modification times and hashes 54972 .PP 54973 The modified time is stored as metadata on the object as 54974 \f[C]X-Object-Meta-Mtime\f[R] as floating point since the epoch accurate 54975 to 1 ns. 54976 .PP 54977 This is a de facto standard (used in the official python-swiftclient 54978 amongst others) for storing the modification time for an object. 54979 .PP 54980 The MD5 hash algorithm is supported. 54981 .SS Restricted filename characters 54982 .PP 54983 .TS 54984 tab(@); 54985 l c c. 54986 T{ 54987 Character 54988 T}@T{ 54989 Value 54990 T}@T{ 54991 Replacement 54992 T} 54993 _ 54994 T{ 54995 NUL 54996 T}@T{ 54997 0x00 54998 T}@T{ 54999 \[u2400] 55000 T} 55001 T{ 55002 / 55003 T}@T{ 55004 0x2F 55005 T}@T{ 55006 \[uFF0F] 55007 T} 55008 .TE 55009 .PP 55010 Invalid UTF-8 bytes will also be 55011 replaced (https://rclone.org/overview/#invalid-utf8), as they can\[aq]t 55012 be used in JSON strings. 55013 .SS Standard options 55014 .PP 55015 Here are the Standard options specific to swift (OpenStack Swift 55016 (Rackspace Cloud Files, Blomp Cloud Storage, Memset Memstore, OVH)). 55017 .SS --swift-env-auth 55018 .PP 55019 Get swift credentials from environment variables in standard OpenStack 55020 form. 55021 .PP 55022 Properties: 55023 .IP \[bu] 2 55024 Config: env_auth 55025 .IP \[bu] 2 55026 Env Var: RCLONE_SWIFT_ENV_AUTH 55027 .IP \[bu] 2 55028 Type: bool 55029 .IP \[bu] 2 55030 Default: false 55031 .IP \[bu] 2 55032 Examples: 55033 .RS 2 55034 .IP \[bu] 2 55035 \[dq]false\[dq] 55036 .RS 2 55037 .IP \[bu] 2 55038 Enter swift credentials in the next step. 55039 .RE 55040 .IP \[bu] 2 55041 \[dq]true\[dq] 55042 .RS 2 55043 .IP \[bu] 2 55044 Get swift credentials from environment vars. 55045 .IP \[bu] 2 55046 Leave other fields blank if using this. 55047 .RE 55048 .RE 55049 .SS --swift-user 55050 .PP 55051 User name to log in (OS_USERNAME). 55052 .PP 55053 Properties: 55054 .IP \[bu] 2 55055 Config: user 55056 .IP \[bu] 2 55057 Env Var: RCLONE_SWIFT_USER 55058 .IP \[bu] 2 55059 Type: string 55060 .IP \[bu] 2 55061 Required: false 55062 .SS --swift-key 55063 .PP 55064 API key or password (OS_PASSWORD). 55065 .PP 55066 Properties: 55067 .IP \[bu] 2 55068 Config: key 55069 .IP \[bu] 2 55070 Env Var: RCLONE_SWIFT_KEY 55071 .IP \[bu] 2 55072 Type: string 55073 .IP \[bu] 2 55074 Required: false 55075 .SS --swift-auth 55076 .PP 55077 Authentication URL for server (OS_AUTH_URL). 55078 .PP 55079 Properties: 55080 .IP \[bu] 2 55081 Config: auth 55082 .IP \[bu] 2 55083 Env Var: RCLONE_SWIFT_AUTH 55084 .IP \[bu] 2 55085 Type: string 55086 .IP \[bu] 2 55087 Required: false 55088 .IP \[bu] 2 55089 Examples: 55090 .RS 2 55091 .IP \[bu] 2 55092 \[dq]https://auth.api.rackspacecloud.com/v1.0\[dq] 55093 .RS 2 55094 .IP \[bu] 2 55095 Rackspace US 55096 .RE 55097 .IP \[bu] 2 55098 \[dq]https://lon.auth.api.rackspacecloud.com/v1.0\[dq] 55099 .RS 2 55100 .IP \[bu] 2 55101 Rackspace UK 55102 .RE 55103 .IP \[bu] 2 55104 \[dq]https://identity.api.rackspacecloud.com/v2.0\[dq] 55105 .RS 2 55106 .IP \[bu] 2 55107 Rackspace v2 55108 .RE 55109 .IP \[bu] 2 55110 \[dq]https://auth.storage.memset.com/v1.0\[dq] 55111 .RS 2 55112 .IP \[bu] 2 55113 Memset Memstore UK 55114 .RE 55115 .IP \[bu] 2 55116 \[dq]https://auth.storage.memset.com/v2.0\[dq] 55117 .RS 2 55118 .IP \[bu] 2 55119 Memset Memstore UK v2 55120 .RE 55121 .IP \[bu] 2 55122 \[dq]https://auth.cloud.ovh.net/v3\[dq] 55123 .RS 2 55124 .IP \[bu] 2 55125 OVH 55126 .RE 55127 .IP \[bu] 2 55128 \[dq]https://authenticate.ain.net\[dq] 55129 .RS 2 55130 .IP \[bu] 2 55131 Blomp Cloud Storage 55132 .RE 55133 .RE 55134 .SS --swift-user-id 55135 .PP 55136 User ID to log in - optional - most swift systems use user and leave 55137 this blank (v3 auth) (OS_USER_ID). 55138 .PP 55139 Properties: 55140 .IP \[bu] 2 55141 Config: user_id 55142 .IP \[bu] 2 55143 Env Var: RCLONE_SWIFT_USER_ID 55144 .IP \[bu] 2 55145 Type: string 55146 .IP \[bu] 2 55147 Required: false 55148 .SS --swift-domain 55149 .PP 55150 User domain - optional (v3 auth) (OS_USER_DOMAIN_NAME) 55151 .PP 55152 Properties: 55153 .IP \[bu] 2 55154 Config: domain 55155 .IP \[bu] 2 55156 Env Var: RCLONE_SWIFT_DOMAIN 55157 .IP \[bu] 2 55158 Type: string 55159 .IP \[bu] 2 55160 Required: false 55161 .SS --swift-tenant 55162 .PP 55163 Tenant name - optional for v1 auth, this or tenant_id required otherwise 55164 (OS_TENANT_NAME or OS_PROJECT_NAME). 55165 .PP 55166 Properties: 55167 .IP \[bu] 2 55168 Config: tenant 55169 .IP \[bu] 2 55170 Env Var: RCLONE_SWIFT_TENANT 55171 .IP \[bu] 2 55172 Type: string 55173 .IP \[bu] 2 55174 Required: false 55175 .SS --swift-tenant-id 55176 .PP 55177 Tenant ID - optional for v1 auth, this or tenant required otherwise 55178 (OS_TENANT_ID). 55179 .PP 55180 Properties: 55181 .IP \[bu] 2 55182 Config: tenant_id 55183 .IP \[bu] 2 55184 Env Var: RCLONE_SWIFT_TENANT_ID 55185 .IP \[bu] 2 55186 Type: string 55187 .IP \[bu] 2 55188 Required: false 55189 .SS --swift-tenant-domain 55190 .PP 55191 Tenant domain - optional (v3 auth) (OS_PROJECT_DOMAIN_NAME). 55192 .PP 55193 Properties: 55194 .IP \[bu] 2 55195 Config: tenant_domain 55196 .IP \[bu] 2 55197 Env Var: RCLONE_SWIFT_TENANT_DOMAIN 55198 .IP \[bu] 2 55199 Type: string 55200 .IP \[bu] 2 55201 Required: false 55202 .SS --swift-region 55203 .PP 55204 Region name - optional (OS_REGION_NAME). 55205 .PP 55206 Properties: 55207 .IP \[bu] 2 55208 Config: region 55209 .IP \[bu] 2 55210 Env Var: RCLONE_SWIFT_REGION 55211 .IP \[bu] 2 55212 Type: string 55213 .IP \[bu] 2 55214 Required: false 55215 .SS --swift-storage-url 55216 .PP 55217 Storage URL - optional (OS_STORAGE_URL). 55218 .PP 55219 Properties: 55220 .IP \[bu] 2 55221 Config: storage_url 55222 .IP \[bu] 2 55223 Env Var: RCLONE_SWIFT_STORAGE_URL 55224 .IP \[bu] 2 55225 Type: string 55226 .IP \[bu] 2 55227 Required: false 55228 .SS --swift-auth-token 55229 .PP 55230 Auth Token from alternate authentication - optional (OS_AUTH_TOKEN). 55231 .PP 55232 Properties: 55233 .IP \[bu] 2 55234 Config: auth_token 55235 .IP \[bu] 2 55236 Env Var: RCLONE_SWIFT_AUTH_TOKEN 55237 .IP \[bu] 2 55238 Type: string 55239 .IP \[bu] 2 55240 Required: false 55241 .SS --swift-application-credential-id 55242 .PP 55243 Application Credential ID (OS_APPLICATION_CREDENTIAL_ID). 55244 .PP 55245 Properties: 55246 .IP \[bu] 2 55247 Config: application_credential_id 55248 .IP \[bu] 2 55249 Env Var: RCLONE_SWIFT_APPLICATION_CREDENTIAL_ID 55250 .IP \[bu] 2 55251 Type: string 55252 .IP \[bu] 2 55253 Required: false 55254 .SS --swift-application-credential-name 55255 .PP 55256 Application Credential Name (OS_APPLICATION_CREDENTIAL_NAME). 55257 .PP 55258 Properties: 55259 .IP \[bu] 2 55260 Config: application_credential_name 55261 .IP \[bu] 2 55262 Env Var: RCLONE_SWIFT_APPLICATION_CREDENTIAL_NAME 55263 .IP \[bu] 2 55264 Type: string 55265 .IP \[bu] 2 55266 Required: false 55267 .SS --swift-application-credential-secret 55268 .PP 55269 Application Credential Secret (OS_APPLICATION_CREDENTIAL_SECRET). 55270 .PP 55271 Properties: 55272 .IP \[bu] 2 55273 Config: application_credential_secret 55274 .IP \[bu] 2 55275 Env Var: RCLONE_SWIFT_APPLICATION_CREDENTIAL_SECRET 55276 .IP \[bu] 2 55277 Type: string 55278 .IP \[bu] 2 55279 Required: false 55280 .SS --swift-auth-version 55281 .PP 55282 AuthVersion - optional - set to (1,2,3) if your auth URL has no version 55283 (ST_AUTH_VERSION). 55284 .PP 55285 Properties: 55286 .IP \[bu] 2 55287 Config: auth_version 55288 .IP \[bu] 2 55289 Env Var: RCLONE_SWIFT_AUTH_VERSION 55290 .IP \[bu] 2 55291 Type: int 55292 .IP \[bu] 2 55293 Default: 0 55294 .SS --swift-endpoint-type 55295 .PP 55296 Endpoint type to choose from the service catalogue (OS_ENDPOINT_TYPE). 55297 .PP 55298 Properties: 55299 .IP \[bu] 2 55300 Config: endpoint_type 55301 .IP \[bu] 2 55302 Env Var: RCLONE_SWIFT_ENDPOINT_TYPE 55303 .IP \[bu] 2 55304 Type: string 55305 .IP \[bu] 2 55306 Default: \[dq]public\[dq] 55307 .IP \[bu] 2 55308 Examples: 55309 .RS 2 55310 .IP \[bu] 2 55311 \[dq]public\[dq] 55312 .RS 2 55313 .IP \[bu] 2 55314 Public (default, choose this if not sure) 55315 .RE 55316 .IP \[bu] 2 55317 \[dq]internal\[dq] 55318 .RS 2 55319 .IP \[bu] 2 55320 Internal (use internal service net) 55321 .RE 55322 .IP \[bu] 2 55323 \[dq]admin\[dq] 55324 .RS 2 55325 .IP \[bu] 2 55326 Admin 55327 .RE 55328 .RE 55329 .SS --swift-storage-policy 55330 .PP 55331 The storage policy to use when creating a new container. 55332 .PP 55333 This applies the specified storage policy when creating a new container. 55334 The policy cannot be changed afterwards. 55335 The allowed configuration values and their meaning depend on your Swift 55336 storage provider. 55337 .PP 55338 Properties: 55339 .IP \[bu] 2 55340 Config: storage_policy 55341 .IP \[bu] 2 55342 Env Var: RCLONE_SWIFT_STORAGE_POLICY 55343 .IP \[bu] 2 55344 Type: string 55345 .IP \[bu] 2 55346 Required: false 55347 .IP \[bu] 2 55348 Examples: 55349 .RS 2 55350 .IP \[bu] 2 55351 \[dq]\[dq] 55352 .RS 2 55353 .IP \[bu] 2 55354 Default 55355 .RE 55356 .IP \[bu] 2 55357 \[dq]pcs\[dq] 55358 .RS 2 55359 .IP \[bu] 2 55360 OVH Public Cloud Storage 55361 .RE 55362 .IP \[bu] 2 55363 \[dq]pca\[dq] 55364 .RS 2 55365 .IP \[bu] 2 55366 OVH Public Cloud Archive 55367 .RE 55368 .RE 55369 .SS Advanced options 55370 .PP 55371 Here are the Advanced options specific to swift (OpenStack Swift 55372 (Rackspace Cloud Files, Blomp Cloud Storage, Memset Memstore, OVH)). 55373 .SS --swift-leave-parts-on-error 55374 .PP 55375 If true avoid calling abort upload on a failure. 55376 .PP 55377 It should be set to true for resuming uploads across different sessions. 55378 .PP 55379 Properties: 55380 .IP \[bu] 2 55381 Config: leave_parts_on_error 55382 .IP \[bu] 2 55383 Env Var: RCLONE_SWIFT_LEAVE_PARTS_ON_ERROR 55384 .IP \[bu] 2 55385 Type: bool 55386 .IP \[bu] 2 55387 Default: false 55388 .SS --swift-chunk-size 55389 .PP 55390 Above this size files will be chunked into a _segments container. 55391 .PP 55392 Above this size files will be chunked into a _segments container. 55393 The default for this is 5 GiB which is its maximum value. 55394 .PP 55395 Properties: 55396 .IP \[bu] 2 55397 Config: chunk_size 55398 .IP \[bu] 2 55399 Env Var: RCLONE_SWIFT_CHUNK_SIZE 55400 .IP \[bu] 2 55401 Type: SizeSuffix 55402 .IP \[bu] 2 55403 Default: 5Gi 55404 .SS --swift-no-chunk 55405 .PP 55406 Don\[aq]t chunk files during streaming upload. 55407 .PP 55408 When doing streaming uploads (e.g. 55409 using rcat or mount) setting this flag will cause the swift backend to 55410 not upload chunked files. 55411 .PP 55412 This will limit the maximum upload size to 5 GiB. 55413 However non chunked files are easier to deal with and have an MD5SUM. 55414 .PP 55415 Rclone will still chunk files bigger than chunk_size when doing normal 55416 copy operations. 55417 .PP 55418 Properties: 55419 .IP \[bu] 2 55420 Config: no_chunk 55421 .IP \[bu] 2 55422 Env Var: RCLONE_SWIFT_NO_CHUNK 55423 .IP \[bu] 2 55424 Type: bool 55425 .IP \[bu] 2 55426 Default: false 55427 .SS --swift-no-large-objects 55428 .PP 55429 Disable support for static and dynamic large objects 55430 .PP 55431 Swift cannot transparently store files bigger than 5 GiB. 55432 There are two schemes for doing that, static or dynamic large objects, 55433 and the API does not allow rclone to determine whether a file is a 55434 static or dynamic large object without doing a HEAD on the object. 55435 Since these need to be treated differently, this means rclone has to 55436 issue HEAD requests for objects for example when reading checksums. 55437 .PP 55438 When \f[C]no_large_objects\f[R] is set, rclone will assume that there 55439 are no static or dynamic large objects stored. 55440 This means it can stop doing the extra HEAD calls which in turn 55441 increases performance greatly especially when doing a swift to swift 55442 transfer with \f[C]--checksum\f[R] set. 55443 .PP 55444 Setting this option implies \f[C]no_chunk\f[R] and also that no files 55445 will be uploaded in chunks, so files bigger than 5 GiB will just fail on 55446 upload. 55447 .PP 55448 If you set this option and there \f[I]are\f[R] static or dynamic large 55449 objects, then this will give incorrect hashes for them. 55450 Downloads will succeed, but other operations such as Remove and Copy 55451 will fail. 55452 .PP 55453 Properties: 55454 .IP \[bu] 2 55455 Config: no_large_objects 55456 .IP \[bu] 2 55457 Env Var: RCLONE_SWIFT_NO_LARGE_OBJECTS 55458 .IP \[bu] 2 55459 Type: bool 55460 .IP \[bu] 2 55461 Default: false 55462 .SS --swift-encoding 55463 .PP 55464 The encoding for the backend. 55465 .PP 55466 See the encoding section in the 55467 overview (https://rclone.org/overview/#encoding) for more info. 55468 .PP 55469 Properties: 55470 .IP \[bu] 2 55471 Config: encoding 55472 .IP \[bu] 2 55473 Env Var: RCLONE_SWIFT_ENCODING 55474 .IP \[bu] 2 55475 Type: Encoding 55476 .IP \[bu] 2 55477 Default: Slash,InvalidUtf8 55478 .SS --swift-description 55479 .PP 55480 Description of the remote 55481 .PP 55482 Properties: 55483 .IP \[bu] 2 55484 Config: description 55485 .IP \[bu] 2 55486 Env Var: RCLONE_SWIFT_DESCRIPTION 55487 .IP \[bu] 2 55488 Type: string 55489 .IP \[bu] 2 55490 Required: false 55491 .SS Limitations 55492 .PP 55493 The Swift API doesn\[aq]t return a correct MD5SUM for segmented files 55494 (Dynamic or Static Large Objects) so rclone won\[aq]t check or use the 55495 MD5SUM for these. 55496 .SS Troubleshooting 55497 .SS Rclone gives Failed to create file system for \[dq]remote:\[dq]: Bad Request 55498 .PP 55499 Due to an oddity of the underlying swift library, it gives a \[dq]Bad 55500 Request\[dq] error rather than a more sensible error when the 55501 authentication fails for Swift. 55502 .PP 55503 So this most likely means your username / password is wrong. 55504 You can investigate further with the \f[C]--dump-bodies\f[R] flag. 55505 .PP 55506 This may also be caused by specifying the region when you shouldn\[aq]t 55507 have (e.g. 55508 OVH). 55509 .SS Rclone gives Failed to create file system: Response didn\[aq]t have storage url and auth token 55510 .PP 55511 This is most likely caused by forgetting to specify your tenant when 55512 setting up a swift remote. 55513 .SS OVH Cloud Archive 55514 .PP 55515 To use rclone with OVH cloud archive, first use \f[C]rclone config\f[R] 55516 to set up a \f[C]swift\f[R] backend with OVH, choosing \f[C]pca\f[R] as 55517 the \f[C]storage_policy\f[R]. 55518 .SS Uploading Objects 55519 .PP 55520 Uploading objects to OVH cloud archive is no different to object 55521 storage, you just simply run the command you like (move, copy or sync) 55522 to upload the objects. 55523 Once uploaded the objects will show in a \[dq]Frozen\[dq] state within 55524 the OVH control panel. 55525 .SS Retrieving Objects 55526 .PP 55527 To retrieve objects use \f[C]rclone copy\f[R] as normal. 55528 If the objects are in a frozen state then rclone will ask for them all 55529 to be unfrozen and it will wait at the end of the output with a message 55530 like the following: 55531 .PP 55532 \f[C]2019/03/23 13:06:33 NOTICE: Received retry after error - sleeping until 2019-03-23T13:16:33.481657164+01:00 (9m59.99985121s)\f[R] 55533 .PP 55534 Rclone will wait for the time specified then retry the copy. 55535 .SH pCloud 55536 .PP 55537 Paths are specified as \f[C]remote:path\f[R] 55538 .PP 55539 Paths may be as deep as required, e.g. 55540 \f[C]remote:directory/subdirectory\f[R]. 55541 .SS Configuration 55542 .PP 55543 The initial setup for pCloud involves getting a token from pCloud which 55544 you need to do in your browser. 55545 \f[C]rclone config\f[R] walks you through it. 55546 .PP 55547 Here is an example of how to make a remote called \f[C]remote\f[R]. 55548 First run: 55549 .IP 55550 .nf 55551 \f[C] 55552 rclone config 55553 \f[R] 55554 .fi 55555 .PP 55556 This will guide you through an interactive setup process: 55557 .IP 55558 .nf 55559 \f[C] 55560 No remotes found, make a new one? 55561 n) New remote 55562 s) Set configuration password 55563 q) Quit config 55564 n/s/q> n 55565 name> remote 55566 Type of storage to configure. 55567 Choose a number from below, or type in your own value 55568 [snip] 55569 XX / Pcloud 55570 \[rs] \[dq]pcloud\[dq] 55571 [snip] 55572 Storage> pcloud 55573 Pcloud App Client Id - leave blank normally. 55574 client_id> 55575 Pcloud App Client Secret - leave blank normally. 55576 client_secret> 55577 Remote config 55578 Use web browser to automatically authenticate rclone with remote? 55579 * Say Y if the machine running rclone has a web browser you can use 55580 * Say N if running rclone on a (remote) machine without web browser access 55581 If not sure try Y. If Y failed, try N. 55582 y) Yes 55583 n) No 55584 y/n> y 55585 If your browser doesn\[aq]t open automatically go to the following link: http://127.0.0.1:53682/auth 55586 Log in and authorize rclone for access 55587 Waiting for code... 55588 Got code 55589 -------------------- 55590 [remote] 55591 client_id = 55592 client_secret = 55593 token = {\[dq]access_token\[dq]:\[dq]XXX\[dq],\[dq]token_type\[dq]:\[dq]bearer\[dq],\[dq]expiry\[dq]:\[dq]0001-01-01T00:00:00Z\[dq]} 55594 -------------------- 55595 y) Yes this is OK 55596 e) Edit this remote 55597 d) Delete this remote 55598 y/e/d> y 55599 \f[R] 55600 .fi 55601 .PP 55602 See the remote setup docs (https://rclone.org/remote_setup/) for how to 55603 set it up on a machine with no Internet browser available. 55604 .PP 55605 Note that rclone runs a webserver on your local machine to collect the 55606 token as returned from pCloud. 55607 This only runs from the moment it opens your browser to the moment you 55608 get back the verification code. 55609 This is on \f[C]http://127.0.0.1:53682/\f[R] and this it may require you 55610 to unblock it temporarily if you are running a host firewall. 55611 .PP 55612 Once configured you can then use \f[C]rclone\f[R] like this, 55613 .PP 55614 List directories in top level of your pCloud 55615 .IP 55616 .nf 55617 \f[C] 55618 rclone lsd remote: 55619 \f[R] 55620 .fi 55621 .PP 55622 List all the files in your pCloud 55623 .IP 55624 .nf 55625 \f[C] 55626 rclone ls remote: 55627 \f[R] 55628 .fi 55629 .PP 55630 To copy a local directory to a pCloud directory called backup 55631 .IP 55632 .nf 55633 \f[C] 55634 rclone copy /home/source remote:backup 55635 \f[R] 55636 .fi 55637 .SS Modification times and hashes 55638 .PP 55639 pCloud allows modification times to be set on objects accurate to 1 55640 second. 55641 These will be used to detect whether objects need syncing or not. 55642 In order to set a Modification time pCloud requires the object be 55643 re-uploaded. 55644 .PP 55645 pCloud supports MD5 and SHA1 hashes in the US region, and SHA1 and 55646 SHA256 hashes in the EU region, so you can use the \f[C]--checksum\f[R] 55647 flag. 55648 .SS Restricted filename characters 55649 .PP 55650 In addition to the default restricted characters 55651 set (https://rclone.org/overview/#restricted-characters) the following 55652 characters are also replaced: 55653 .PP 55654 .TS 55655 tab(@); 55656 l c c. 55657 T{ 55658 Character 55659 T}@T{ 55660 Value 55661 T}@T{ 55662 Replacement 55663 T} 55664 _ 55665 T{ 55666 \[rs] 55667 T}@T{ 55668 0x5C 55669 T}@T{ 55670 \[uFF3C] 55671 T} 55672 .TE 55673 .PP 55674 Invalid UTF-8 bytes will also be 55675 replaced (https://rclone.org/overview/#invalid-utf8), as they can\[aq]t 55676 be used in JSON strings. 55677 .SS Deleting files 55678 .PP 55679 Deleted files will be moved to the trash. 55680 Your subscription level will determine how long items stay in the trash. 55681 \f[C]rclone cleanup\f[R] can be used to empty the trash. 55682 .SS Emptying the trash 55683 .PP 55684 Due to an API limitation, the \f[C]rclone cleanup\f[R] command will only 55685 work if you set your username and password in the advanced options for 55686 this backend. 55687 Since we generally want to avoid storing user passwords in the rclone 55688 config file, we advise you to only set this up if you need the 55689 \f[C]rclone cleanup\f[R] command to work. 55690 .SS Root folder ID 55691 .PP 55692 You can set the \f[C]root_folder_id\f[R] for rclone. 55693 This is the directory (identified by its \f[C]Folder ID\f[R]) that 55694 rclone considers to be the root of your pCloud drive. 55695 .PP 55696 Normally you will leave this blank and rclone will determine the correct 55697 root to use itself. 55698 .PP 55699 However you can set this to restrict rclone to a specific folder 55700 hierarchy. 55701 .PP 55702 In order to do this you will have to find the \f[C]Folder ID\f[R] of the 55703 directory you wish rclone to display. 55704 This will be the \f[C]folder\f[R] field of the URL when you open the 55705 relevant folder in the pCloud web interface. 55706 .PP 55707 So if the folder you want rclone to use has a URL which looks like 55708 \f[C]https://my.pcloud.com/#page=filemanager&folder=5xxxxxxxx8&tpl=foldergrid\f[R] 55709 in the browser, then you use \f[C]5xxxxxxxx8\f[R] as the 55710 \f[C]root_folder_id\f[R] in the config. 55711 .SS Standard options 55712 .PP 55713 Here are the Standard options specific to pcloud (Pcloud). 55714 .SS --pcloud-client-id 55715 .PP 55716 OAuth Client Id. 55717 .PP 55718 Leave blank normally. 55719 .PP 55720 Properties: 55721 .IP \[bu] 2 55722 Config: client_id 55723 .IP \[bu] 2 55724 Env Var: RCLONE_PCLOUD_CLIENT_ID 55725 .IP \[bu] 2 55726 Type: string 55727 .IP \[bu] 2 55728 Required: false 55729 .SS --pcloud-client-secret 55730 .PP 55731 OAuth Client Secret. 55732 .PP 55733 Leave blank normally. 55734 .PP 55735 Properties: 55736 .IP \[bu] 2 55737 Config: client_secret 55738 .IP \[bu] 2 55739 Env Var: RCLONE_PCLOUD_CLIENT_SECRET 55740 .IP \[bu] 2 55741 Type: string 55742 .IP \[bu] 2 55743 Required: false 55744 .SS Advanced options 55745 .PP 55746 Here are the Advanced options specific to pcloud (Pcloud). 55747 .SS --pcloud-token 55748 .PP 55749 OAuth Access Token as a JSON blob. 55750 .PP 55751 Properties: 55752 .IP \[bu] 2 55753 Config: token 55754 .IP \[bu] 2 55755 Env Var: RCLONE_PCLOUD_TOKEN 55756 .IP \[bu] 2 55757 Type: string 55758 .IP \[bu] 2 55759 Required: false 55760 .SS --pcloud-auth-url 55761 .PP 55762 Auth server URL. 55763 .PP 55764 Leave blank to use the provider defaults. 55765 .PP 55766 Properties: 55767 .IP \[bu] 2 55768 Config: auth_url 55769 .IP \[bu] 2 55770 Env Var: RCLONE_PCLOUD_AUTH_URL 55771 .IP \[bu] 2 55772 Type: string 55773 .IP \[bu] 2 55774 Required: false 55775 .SS --pcloud-token-url 55776 .PP 55777 Token server url. 55778 .PP 55779 Leave blank to use the provider defaults. 55780 .PP 55781 Properties: 55782 .IP \[bu] 2 55783 Config: token_url 55784 .IP \[bu] 2 55785 Env Var: RCLONE_PCLOUD_TOKEN_URL 55786 .IP \[bu] 2 55787 Type: string 55788 .IP \[bu] 2 55789 Required: false 55790 .SS --pcloud-encoding 55791 .PP 55792 The encoding for the backend. 55793 .PP 55794 See the encoding section in the 55795 overview (https://rclone.org/overview/#encoding) for more info. 55796 .PP 55797 Properties: 55798 .IP \[bu] 2 55799 Config: encoding 55800 .IP \[bu] 2 55801 Env Var: RCLONE_PCLOUD_ENCODING 55802 .IP \[bu] 2 55803 Type: Encoding 55804 .IP \[bu] 2 55805 Default: Slash,BackSlash,Del,Ctl,InvalidUtf8,Dot 55806 .SS --pcloud-root-folder-id 55807 .PP 55808 Fill in for rclone to use a non root folder as its starting point. 55809 .PP 55810 Properties: 55811 .IP \[bu] 2 55812 Config: root_folder_id 55813 .IP \[bu] 2 55814 Env Var: RCLONE_PCLOUD_ROOT_FOLDER_ID 55815 .IP \[bu] 2 55816 Type: string 55817 .IP \[bu] 2 55818 Default: \[dq]d0\[dq] 55819 .SS --pcloud-hostname 55820 .PP 55821 Hostname to connect to. 55822 .PP 55823 This is normally set when rclone initially does the oauth connection, 55824 however you will need to set it by hand if you are using remote config 55825 with rclone authorize. 55826 .PP 55827 Properties: 55828 .IP \[bu] 2 55829 Config: hostname 55830 .IP \[bu] 2 55831 Env Var: RCLONE_PCLOUD_HOSTNAME 55832 .IP \[bu] 2 55833 Type: string 55834 .IP \[bu] 2 55835 Default: \[dq]api.pcloud.com\[dq] 55836 .IP \[bu] 2 55837 Examples: 55838 .RS 2 55839 .IP \[bu] 2 55840 \[dq]api.pcloud.com\[dq] 55841 .RS 2 55842 .IP \[bu] 2 55843 Original/US region 55844 .RE 55845 .IP \[bu] 2 55846 \[dq]eapi.pcloud.com\[dq] 55847 .RS 2 55848 .IP \[bu] 2 55849 EU region 55850 .RE 55851 .RE 55852 .SS --pcloud-username 55853 .PP 55854 Your pcloud username. 55855 .PP 55856 This is only required when you want to use the cleanup command. 55857 Due to a bug in the pcloud API the required API does not support OAuth 55858 authentication so we have to rely on user password authentication for 55859 it. 55860 .PP 55861 Properties: 55862 .IP \[bu] 2 55863 Config: username 55864 .IP \[bu] 2 55865 Env Var: RCLONE_PCLOUD_USERNAME 55866 .IP \[bu] 2 55867 Type: string 55868 .IP \[bu] 2 55869 Required: false 55870 .SS --pcloud-password 55871 .PP 55872 Your pcloud password. 55873 .PP 55874 \f[B]NB\f[R] Input to this must be obscured - see rclone 55875 obscure (https://rclone.org/commands/rclone_obscure/). 55876 .PP 55877 Properties: 55878 .IP \[bu] 2 55879 Config: password 55880 .IP \[bu] 2 55881 Env Var: RCLONE_PCLOUD_PASSWORD 55882 .IP \[bu] 2 55883 Type: string 55884 .IP \[bu] 2 55885 Required: false 55886 .SS --pcloud-description 55887 .PP 55888 Description of the remote 55889 .PP 55890 Properties: 55891 .IP \[bu] 2 55892 Config: description 55893 .IP \[bu] 2 55894 Env Var: RCLONE_PCLOUD_DESCRIPTION 55895 .IP \[bu] 2 55896 Type: string 55897 .IP \[bu] 2 55898 Required: false 55899 .SH PikPak 55900 .PP 55901 PikPak is a private cloud drive (https://mypikpak.com/). 55902 .PP 55903 Paths are specified as \f[C]remote:path\f[R], and may be as deep as 55904 required, e.g. 55905 \f[C]remote:directory/subdirectory\f[R]. 55906 .SS Configuration 55907 .PP 55908 Here is an example of making a remote for PikPak. 55909 .PP 55910 First run: 55911 .IP 55912 .nf 55913 \f[C] 55914 rclone config 55915 \f[R] 55916 .fi 55917 .PP 55918 This will guide you through an interactive setup process: 55919 .IP 55920 .nf 55921 \f[C] 55922 No remotes found, make a new one? 55923 n) New remote 55924 s) Set configuration password 55925 q) Quit config 55926 n/s/q> n 55927 55928 Enter name for new remote. 55929 name> remote 55930 55931 Option Storage. 55932 Type of storage to configure. 55933 Choose a number from below, or type in your own value. 55934 XX / PikPak 55935 \[rs] (pikpak) 55936 Storage> XX 55937 55938 Option user. 55939 Pikpak username. 55940 Enter a value. 55941 user> USERNAME 55942 55943 Option pass. 55944 Pikpak password. 55945 Choose an alternative below. 55946 y) Yes, type in my own password 55947 g) Generate random password 55948 y/g> y 55949 Enter the password: 55950 password: 55951 Confirm the password: 55952 password: 55953 55954 Edit advanced config? 55955 y) Yes 55956 n) No (default) 55957 y/n> 55958 55959 Configuration complete. 55960 Options: 55961 - type: pikpak 55962 - user: USERNAME 55963 - pass: *** ENCRYPTED *** 55964 - token: {\[dq]access_token\[dq]:\[dq]eyJ...\[dq],\[dq]token_type\[dq]:\[dq]Bearer\[dq],\[dq]refresh_token\[dq]:\[dq]os...\[dq],\[dq]expiry\[dq]:\[dq]2023-01-26T18:54:32.170582647+09:00\[dq]} 55965 Keep this \[dq]remote\[dq] remote? 55966 y) Yes this is OK (default) 55967 e) Edit this remote 55968 d) Delete this remote 55969 y/e/d> y 55970 \f[R] 55971 .fi 55972 .SS Modification times and hashes 55973 .PP 55974 PikPak keeps modification times on objects, and updates them when 55975 uploading objects, but it does not support changing only the 55976 modification time 55977 .PP 55978 The MD5 hash algorithm is supported. 55979 .SS Standard options 55980 .PP 55981 Here are the Standard options specific to pikpak (PikPak). 55982 .SS --pikpak-user 55983 .PP 55984 Pikpak username. 55985 .PP 55986 Properties: 55987 .IP \[bu] 2 55988 Config: user 55989 .IP \[bu] 2 55990 Env Var: RCLONE_PIKPAK_USER 55991 .IP \[bu] 2 55992 Type: string 55993 .IP \[bu] 2 55994 Required: true 55995 .SS --pikpak-pass 55996 .PP 55997 Pikpak password. 55998 .PP 55999 \f[B]NB\f[R] Input to this must be obscured - see rclone 56000 obscure (https://rclone.org/commands/rclone_obscure/). 56001 .PP 56002 Properties: 56003 .IP \[bu] 2 56004 Config: pass 56005 .IP \[bu] 2 56006 Env Var: RCLONE_PIKPAK_PASS 56007 .IP \[bu] 2 56008 Type: string 56009 .IP \[bu] 2 56010 Required: true 56011 .SS Advanced options 56012 .PP 56013 Here are the Advanced options specific to pikpak (PikPak). 56014 .SS --pikpak-client-id 56015 .PP 56016 OAuth Client Id. 56017 .PP 56018 Leave blank normally. 56019 .PP 56020 Properties: 56021 .IP \[bu] 2 56022 Config: client_id 56023 .IP \[bu] 2 56024 Env Var: RCLONE_PIKPAK_CLIENT_ID 56025 .IP \[bu] 2 56026 Type: string 56027 .IP \[bu] 2 56028 Required: false 56029 .SS --pikpak-client-secret 56030 .PP 56031 OAuth Client Secret. 56032 .PP 56033 Leave blank normally. 56034 .PP 56035 Properties: 56036 .IP \[bu] 2 56037 Config: client_secret 56038 .IP \[bu] 2 56039 Env Var: RCLONE_PIKPAK_CLIENT_SECRET 56040 .IP \[bu] 2 56041 Type: string 56042 .IP \[bu] 2 56043 Required: false 56044 .SS --pikpak-token 56045 .PP 56046 OAuth Access Token as a JSON blob. 56047 .PP 56048 Properties: 56049 .IP \[bu] 2 56050 Config: token 56051 .IP \[bu] 2 56052 Env Var: RCLONE_PIKPAK_TOKEN 56053 .IP \[bu] 2 56054 Type: string 56055 .IP \[bu] 2 56056 Required: false 56057 .SS --pikpak-auth-url 56058 .PP 56059 Auth server URL. 56060 .PP 56061 Leave blank to use the provider defaults. 56062 .PP 56063 Properties: 56064 .IP \[bu] 2 56065 Config: auth_url 56066 .IP \[bu] 2 56067 Env Var: RCLONE_PIKPAK_AUTH_URL 56068 .IP \[bu] 2 56069 Type: string 56070 .IP \[bu] 2 56071 Required: false 56072 .SS --pikpak-token-url 56073 .PP 56074 Token server url. 56075 .PP 56076 Leave blank to use the provider defaults. 56077 .PP 56078 Properties: 56079 .IP \[bu] 2 56080 Config: token_url 56081 .IP \[bu] 2 56082 Env Var: RCLONE_PIKPAK_TOKEN_URL 56083 .IP \[bu] 2 56084 Type: string 56085 .IP \[bu] 2 56086 Required: false 56087 .SS --pikpak-root-folder-id 56088 .PP 56089 ID of the root folder. 56090 Leave blank normally. 56091 .PP 56092 Fill in for rclone to use a non root folder as its starting point. 56093 .PP 56094 Properties: 56095 .IP \[bu] 2 56096 Config: root_folder_id 56097 .IP \[bu] 2 56098 Env Var: RCLONE_PIKPAK_ROOT_FOLDER_ID 56099 .IP \[bu] 2 56100 Type: string 56101 .IP \[bu] 2 56102 Required: false 56103 .SS --pikpak-use-trash 56104 .PP 56105 Send files to the trash instead of deleting permanently. 56106 .PP 56107 Defaults to true, namely sending files to the trash. 56108 Use \f[C]--pikpak-use-trash=false\f[R] to delete files permanently 56109 instead. 56110 .PP 56111 Properties: 56112 .IP \[bu] 2 56113 Config: use_trash 56114 .IP \[bu] 2 56115 Env Var: RCLONE_PIKPAK_USE_TRASH 56116 .IP \[bu] 2 56117 Type: bool 56118 .IP \[bu] 2 56119 Default: true 56120 .SS --pikpak-trashed-only 56121 .PP 56122 Only show files that are in the trash. 56123 .PP 56124 This will show trashed files in their original directory structure. 56125 .PP 56126 Properties: 56127 .IP \[bu] 2 56128 Config: trashed_only 56129 .IP \[bu] 2 56130 Env Var: RCLONE_PIKPAK_TRASHED_ONLY 56131 .IP \[bu] 2 56132 Type: bool 56133 .IP \[bu] 2 56134 Default: false 56135 .SS --pikpak-hash-memory-limit 56136 .PP 56137 Files bigger than this will be cached on disk to calculate hash if 56138 required. 56139 .PP 56140 Properties: 56141 .IP \[bu] 2 56142 Config: hash_memory_limit 56143 .IP \[bu] 2 56144 Env Var: RCLONE_PIKPAK_HASH_MEMORY_LIMIT 56145 .IP \[bu] 2 56146 Type: SizeSuffix 56147 .IP \[bu] 2 56148 Default: 10Mi 56149 .SS --pikpak-encoding 56150 .PP 56151 The encoding for the backend. 56152 .PP 56153 See the encoding section in the 56154 overview (https://rclone.org/overview/#encoding) for more info. 56155 .PP 56156 Properties: 56157 .IP \[bu] 2 56158 Config: encoding 56159 .IP \[bu] 2 56160 Env Var: RCLONE_PIKPAK_ENCODING 56161 .IP \[bu] 2 56162 Type: Encoding 56163 .IP \[bu] 2 56164 Default: 56165 Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Ctl,LeftSpace,RightSpace,RightPeriod,InvalidUtf8,Dot 56166 .SS --pikpak-description 56167 .PP 56168 Description of the remote 56169 .PP 56170 Properties: 56171 .IP \[bu] 2 56172 Config: description 56173 .IP \[bu] 2 56174 Env Var: RCLONE_PIKPAK_DESCRIPTION 56175 .IP \[bu] 2 56176 Type: string 56177 .IP \[bu] 2 56178 Required: false 56179 .SS Backend commands 56180 .PP 56181 Here are the commands specific to the pikpak backend. 56182 .PP 56183 Run them with 56184 .IP 56185 .nf 56186 \f[C] 56187 rclone backend COMMAND remote: 56188 \f[R] 56189 .fi 56190 .PP 56191 The help below will explain what arguments each command takes. 56192 .PP 56193 See the backend (https://rclone.org/commands/rclone_backend/) command 56194 for more info on how to pass options and arguments. 56195 .PP 56196 These can be run on a running backend using the rc command 56197 backend/command (https://rclone.org/rc/#backend-command). 56198 .SS addurl 56199 .PP 56200 Add offline download task for url 56201 .IP 56202 .nf 56203 \f[C] 56204 rclone backend addurl remote: [options] [<arguments>+] 56205 \f[R] 56206 .fi 56207 .PP 56208 This command adds offline download task for url. 56209 .PP 56210 Usage: 56211 .IP 56212 .nf 56213 \f[C] 56214 rclone backend addurl pikpak:dirpath url 56215 \f[R] 56216 .fi 56217 .PP 56218 Downloads will be stored in \[aq]dirpath\[aq]. 56219 If \[aq]dirpath\[aq] is invalid, download will fallback to default 56220 \[aq]My Pack\[aq] folder. 56221 .SS decompress 56222 .PP 56223 Request decompress of a file/files in a folder 56224 .IP 56225 .nf 56226 \f[C] 56227 rclone backend decompress remote: [options] [<arguments>+] 56228 \f[R] 56229 .fi 56230 .PP 56231 This command requests decompress of file/files in a folder. 56232 .PP 56233 Usage: 56234 .IP 56235 .nf 56236 \f[C] 56237 rclone backend decompress pikpak:dirpath {filename} -o password=password 56238 rclone backend decompress pikpak:dirpath {filename} -o delete-src-file 56239 \f[R] 56240 .fi 56241 .PP 56242 An optional argument \[aq]filename\[aq] can be specified for a file 56243 located in \[aq]pikpak:dirpath\[aq]. 56244 You may want to pass \[aq]-o password=password\[aq] for a 56245 password-protected files. 56246 Also, pass \[aq]-o delete-src-file\[aq] to delete source files after 56247 decompression finished. 56248 .PP 56249 Result: 56250 .IP 56251 .nf 56252 \f[C] 56253 { 56254 \[dq]Decompressed\[dq]: 17, 56255 \[dq]SourceDeleted\[dq]: 0, 56256 \[dq]Errors\[dq]: 0 56257 } 56258 \f[R] 56259 .fi 56260 .SS Limitations 56261 .SS Hashes may be empty 56262 .PP 56263 PikPak supports MD5 hash, but sometimes given empty especially for 56264 user-uploaded files. 56265 .SS Deleted files still visible with trashed-only 56266 .PP 56267 Deleted files will still be visible with \f[C]--pikpak-trashed-only\f[R] 56268 even after the trash emptied. 56269 This goes away after few days. 56270 .SH premiumize.me 56271 .PP 56272 Paths are specified as \f[C]remote:path\f[R] 56273 .PP 56274 Paths may be as deep as required, e.g. 56275 \f[C]remote:directory/subdirectory\f[R]. 56276 .SS Configuration 56277 .PP 56278 The initial setup for premiumize.me (https://premiumize.me/) involves 56279 getting a token from premiumize.me which you need to do in your browser. 56280 \f[C]rclone config\f[R] walks you through it. 56281 .PP 56282 Here is an example of how to make a remote called \f[C]remote\f[R]. 56283 First run: 56284 .IP 56285 .nf 56286 \f[C] 56287 rclone config 56288 \f[R] 56289 .fi 56290 .PP 56291 This will guide you through an interactive setup process: 56292 .IP 56293 .nf 56294 \f[C] 56295 No remotes found, make a new one? 56296 n) New remote 56297 s) Set configuration password 56298 q) Quit config 56299 n/s/q> n 56300 name> remote 56301 Type of storage to configure. 56302 Enter a string value. Press Enter for the default (\[dq]\[dq]). 56303 Choose a number from below, or type in your own value 56304 [snip] 56305 XX / premiumize.me 56306 \[rs] \[dq]premiumizeme\[dq] 56307 [snip] 56308 Storage> premiumizeme 56309 ** See help for premiumizeme backend at: https://rclone.org/premiumizeme/ ** 56310 56311 Remote config 56312 Use web browser to automatically authenticate rclone with remote? 56313 * Say Y if the machine running rclone has a web browser you can use 56314 * Say N if running rclone on a (remote) machine without web browser access 56315 If not sure try Y. If Y failed, try N. 56316 y) Yes 56317 n) No 56318 y/n> y 56319 If your browser doesn\[aq]t open automatically go to the following link: http://127.0.0.1:53682/auth 56320 Log in and authorize rclone for access 56321 Waiting for code... 56322 Got code 56323 -------------------- 56324 [remote] 56325 type = premiumizeme 56326 token = {\[dq]access_token\[dq]:\[dq]XXX\[dq],\[dq]token_type\[dq]:\[dq]Bearer\[dq],\[dq]refresh_token\[dq]:\[dq]XXX\[dq],\[dq]expiry\[dq]:\[dq]2029-08-07T18:44:15.548915378+01:00\[dq]} 56327 -------------------- 56328 y) Yes this is OK 56329 e) Edit this remote 56330 d) Delete this remote 56331 y/e/d> 56332 \f[R] 56333 .fi 56334 .PP 56335 See the remote setup docs (https://rclone.org/remote_setup/) for how to 56336 set it up on a machine with no Internet browser available. 56337 .PP 56338 Note that rclone runs a webserver on your local machine to collect the 56339 token as returned from premiumize.me. 56340 This only runs from the moment it opens your browser to the moment you 56341 get back the verification code. 56342 This is on \f[C]http://127.0.0.1:53682/\f[R] and this it may require you 56343 to unblock it temporarily if you are running a host firewall. 56344 .PP 56345 Once configured you can then use \f[C]rclone\f[R] like this, 56346 .PP 56347 List directories in top level of your premiumize.me 56348 .IP 56349 .nf 56350 \f[C] 56351 rclone lsd remote: 56352 \f[R] 56353 .fi 56354 .PP 56355 List all the files in your premiumize.me 56356 .IP 56357 .nf 56358 \f[C] 56359 rclone ls remote: 56360 \f[R] 56361 .fi 56362 .PP 56363 To copy a local directory to an premiumize.me directory called backup 56364 .IP 56365 .nf 56366 \f[C] 56367 rclone copy /home/source remote:backup 56368 \f[R] 56369 .fi 56370 .SS Modification times and hashes 56371 .PP 56372 premiumize.me does not support modification times or hashes, therefore 56373 syncing will default to \f[C]--size-only\f[R] checking. 56374 Note that using \f[C]--update\f[R] will work. 56375 .SS Restricted filename characters 56376 .PP 56377 In addition to the default restricted characters 56378 set (https://rclone.org/overview/#restricted-characters) the following 56379 characters are also replaced: 56380 .PP 56381 .TS 56382 tab(@); 56383 l c c. 56384 T{ 56385 Character 56386 T}@T{ 56387 Value 56388 T}@T{ 56389 Replacement 56390 T} 56391 _ 56392 T{ 56393 \[rs] 56394 T}@T{ 56395 0x5C 56396 T}@T{ 56397 \[uFF3C] 56398 T} 56399 T{ 56400 \[dq] 56401 T}@T{ 56402 0x22 56403 T}@T{ 56404 \[uFF02] 56405 T} 56406 .TE 56407 .PP 56408 Invalid UTF-8 bytes will also be 56409 replaced (https://rclone.org/overview/#invalid-utf8), as they can\[aq]t 56410 be used in JSON strings. 56411 .SS Standard options 56412 .PP 56413 Here are the Standard options specific to premiumizeme (premiumize.me). 56414 .SS --premiumizeme-client-id 56415 .PP 56416 OAuth Client Id. 56417 .PP 56418 Leave blank normally. 56419 .PP 56420 Properties: 56421 .IP \[bu] 2 56422 Config: client_id 56423 .IP \[bu] 2 56424 Env Var: RCLONE_PREMIUMIZEME_CLIENT_ID 56425 .IP \[bu] 2 56426 Type: string 56427 .IP \[bu] 2 56428 Required: false 56429 .SS --premiumizeme-client-secret 56430 .PP 56431 OAuth Client Secret. 56432 .PP 56433 Leave blank normally. 56434 .PP 56435 Properties: 56436 .IP \[bu] 2 56437 Config: client_secret 56438 .IP \[bu] 2 56439 Env Var: RCLONE_PREMIUMIZEME_CLIENT_SECRET 56440 .IP \[bu] 2 56441 Type: string 56442 .IP \[bu] 2 56443 Required: false 56444 .SS --premiumizeme-api-key 56445 .PP 56446 API Key. 56447 .PP 56448 This is not normally used - use oauth instead. 56449 .PP 56450 Properties: 56451 .IP \[bu] 2 56452 Config: api_key 56453 .IP \[bu] 2 56454 Env Var: RCLONE_PREMIUMIZEME_API_KEY 56455 .IP \[bu] 2 56456 Type: string 56457 .IP \[bu] 2 56458 Required: false 56459 .SS Advanced options 56460 .PP 56461 Here are the Advanced options specific to premiumizeme (premiumize.me). 56462 .SS --premiumizeme-token 56463 .PP 56464 OAuth Access Token as a JSON blob. 56465 .PP 56466 Properties: 56467 .IP \[bu] 2 56468 Config: token 56469 .IP \[bu] 2 56470 Env Var: RCLONE_PREMIUMIZEME_TOKEN 56471 .IP \[bu] 2 56472 Type: string 56473 .IP \[bu] 2 56474 Required: false 56475 .SS --premiumizeme-auth-url 56476 .PP 56477 Auth server URL. 56478 .PP 56479 Leave blank to use the provider defaults. 56480 .PP 56481 Properties: 56482 .IP \[bu] 2 56483 Config: auth_url 56484 .IP \[bu] 2 56485 Env Var: RCLONE_PREMIUMIZEME_AUTH_URL 56486 .IP \[bu] 2 56487 Type: string 56488 .IP \[bu] 2 56489 Required: false 56490 .SS --premiumizeme-token-url 56491 .PP 56492 Token server url. 56493 .PP 56494 Leave blank to use the provider defaults. 56495 .PP 56496 Properties: 56497 .IP \[bu] 2 56498 Config: token_url 56499 .IP \[bu] 2 56500 Env Var: RCLONE_PREMIUMIZEME_TOKEN_URL 56501 .IP \[bu] 2 56502 Type: string 56503 .IP \[bu] 2 56504 Required: false 56505 .SS --premiumizeme-encoding 56506 .PP 56507 The encoding for the backend. 56508 .PP 56509 See the encoding section in the 56510 overview (https://rclone.org/overview/#encoding) for more info. 56511 .PP 56512 Properties: 56513 .IP \[bu] 2 56514 Config: encoding 56515 .IP \[bu] 2 56516 Env Var: RCLONE_PREMIUMIZEME_ENCODING 56517 .IP \[bu] 2 56518 Type: Encoding 56519 .IP \[bu] 2 56520 Default: Slash,DoubleQuote,BackSlash,Del,Ctl,InvalidUtf8,Dot 56521 .SS --premiumizeme-description 56522 .PP 56523 Description of the remote 56524 .PP 56525 Properties: 56526 .IP \[bu] 2 56527 Config: description 56528 .IP \[bu] 2 56529 Env Var: RCLONE_PREMIUMIZEME_DESCRIPTION 56530 .IP \[bu] 2 56531 Type: string 56532 .IP \[bu] 2 56533 Required: false 56534 .SS Limitations 56535 .PP 56536 Note that premiumize.me is case insensitive so you can\[aq]t have a file 56537 called \[dq]Hello.doc\[dq] and one called \[dq]hello.doc\[dq]. 56538 .PP 56539 premiumize.me file names can\[aq]t have the \f[C]\[rs]\f[R] or 56540 \f[C]\[dq]\f[R] characters in. 56541 rclone maps these to and from an identical looking unicode equivalents 56542 \f[C]\[uFF3C]\f[R] and \f[C]\[uFF02]\f[R] 56543 .PP 56544 premiumize.me only supports filenames up to 255 characters in length. 56545 .SH Proton Drive 56546 .PP 56547 Proton Drive (https://proton.me/drive) is an end-to-end encrypted Swiss 56548 vault for your files that protects your data. 56549 .PP 56550 This is an rclone backend for Proton Drive which supports the file 56551 transfer features of Proton Drive using the same client-side encryption. 56552 .PP 56553 Due to the fact that Proton Drive doesn\[aq]t publish its API 56554 documentation, this backend is implemented with best efforts by reading 56555 the open-sourced client source code and observing the Proton Drive 56556 traffic in the browser. 56557 .PP 56558 \f[B]NB\f[R] This backend is currently in Beta. 56559 It is believed to be correct and all the integration tests pass. 56560 However the Proton Drive protocol has evolved over time there may be 56561 accounts it is not compatible with. 56562 Please post on the rclone forum (https://forum.rclone.org/) if you find 56563 an incompatibility. 56564 .PP 56565 Paths are specified as \f[C]remote:path\f[R] 56566 .PP 56567 Paths may be as deep as required, e.g. 56568 \f[C]remote:directory/subdirectory\f[R]. 56569 .SS Configurations 56570 .PP 56571 Here is an example of how to make a remote called \f[C]remote\f[R]. 56572 First run: 56573 .IP 56574 .nf 56575 \f[C] 56576 rclone config 56577 \f[R] 56578 .fi 56579 .PP 56580 This will guide you through an interactive setup process: 56581 .IP 56582 .nf 56583 \f[C] 56584 No remotes found, make a new one? 56585 n) New remote 56586 s) Set configuration password 56587 q) Quit config 56588 n/s/q> n 56589 name> remote 56590 Type of storage to configure. 56591 Choose a number from below, or type in your own value 56592 [snip] 56593 XX / Proton Drive 56594 \[rs] \[dq]Proton Drive\[dq] 56595 [snip] 56596 Storage> protondrive 56597 User name 56598 user> you\[at]protonmail.com 56599 Password. 56600 y) Yes type in my own password 56601 g) Generate random password 56602 n) No leave this optional password blank 56603 y/g/n> y 56604 Enter the password: 56605 password: 56606 Confirm the password: 56607 password: 56608 Option 2fa. 56609 2FA code (if the account requires one) 56610 Enter a value. Press Enter to leave empty. 56611 2fa> 123456 56612 Remote config 56613 -------------------- 56614 [remote] 56615 type = protondrive 56616 user = you\[at]protonmail.com 56617 pass = *** ENCRYPTED *** 56618 -------------------- 56619 y) Yes this is OK 56620 e) Edit this remote 56621 d) Delete this remote 56622 y/e/d> y 56623 \f[R] 56624 .fi 56625 .PP 56626 \f[B]NOTE:\f[R] The Proton Drive encryption keys need to have been 56627 already generated after a regular login via the browser, otherwise 56628 attempting to use the credentials in \f[C]rclone\f[R] will fail. 56629 .PP 56630 Once configured you can then use \f[C]rclone\f[R] like this, 56631 .PP 56632 List directories in top level of your Proton Drive 56633 .IP 56634 .nf 56635 \f[C] 56636 rclone lsd remote: 56637 \f[R] 56638 .fi 56639 .PP 56640 List all the files in your Proton Drive 56641 .IP 56642 .nf 56643 \f[C] 56644 rclone ls remote: 56645 \f[R] 56646 .fi 56647 .PP 56648 To copy a local directory to an Proton Drive directory called backup 56649 .IP 56650 .nf 56651 \f[C] 56652 rclone copy /home/source remote:backup 56653 \f[R] 56654 .fi 56655 .SS Modification times and hashes 56656 .PP 56657 Proton Drive Bridge does not support updating modification times yet. 56658 .PP 56659 The SHA1 hash algorithm is supported. 56660 .SS Restricted filename characters 56661 .PP 56662 Invalid UTF-8 bytes will be 56663 replaced (https://rclone.org/overview/#invalid-utf8), also left and 56664 right spaces will be removed (code 56665 reference (https://github.com/ProtonMail/WebClients/blob/b4eba99d241af4fdae06ff7138bd651a40ef5d3c/applications/drive/src/app/store/_links/validation.ts#L51)) 56666 .SS Duplicated files 56667 .PP 56668 Proton Drive can not have two files with exactly the same name and path. 56669 If the conflict occurs, depending on the advanced config, the file might 56670 or might not be overwritten. 56671 .SS Mailbox password (https://proton.me/support/the-difference-between-the-mailbox-password-and-login-password) 56672 .PP 56673 Please set your mailbox password in the advanced config section. 56674 .SS Caching 56675 .PP 56676 The cache is currently built for the case when the rclone is the only 56677 instance performing operations to the mount point. 56678 The event system, which is the proton API system that provides 56679 visibility of what has changed on the drive, is yet to be implemented, 56680 so updates from other clients won\[cq]t be reflected in the cache. 56681 Thus, if there are concurrent clients accessing the same mount point, 56682 then we might have a problem with caching the stale data. 56683 .SS Standard options 56684 .PP 56685 Here are the Standard options specific to protondrive (Proton Drive). 56686 .SS --protondrive-username 56687 .PP 56688 The username of your proton account 56689 .PP 56690 Properties: 56691 .IP \[bu] 2 56692 Config: username 56693 .IP \[bu] 2 56694 Env Var: RCLONE_PROTONDRIVE_USERNAME 56695 .IP \[bu] 2 56696 Type: string 56697 .IP \[bu] 2 56698 Required: true 56699 .SS --protondrive-password 56700 .PP 56701 The password of your proton account. 56702 .PP 56703 \f[B]NB\f[R] Input to this must be obscured - see rclone 56704 obscure (https://rclone.org/commands/rclone_obscure/). 56705 .PP 56706 Properties: 56707 .IP \[bu] 2 56708 Config: password 56709 .IP \[bu] 2 56710 Env Var: RCLONE_PROTONDRIVE_PASSWORD 56711 .IP \[bu] 2 56712 Type: string 56713 .IP \[bu] 2 56714 Required: true 56715 .SS --protondrive-2fa 56716 .PP 56717 The 2FA code 56718 .PP 56719 The value can also be provided with --protondrive-2fa=000000 56720 .PP 56721 The 2FA code of your proton drive account if the account is set up with 56722 two-factor authentication 56723 .PP 56724 Properties: 56725 .IP \[bu] 2 56726 Config: 2fa 56727 .IP \[bu] 2 56728 Env Var: RCLONE_PROTONDRIVE_2FA 56729 .IP \[bu] 2 56730 Type: string 56731 .IP \[bu] 2 56732 Required: false 56733 .SS Advanced options 56734 .PP 56735 Here are the Advanced options specific to protondrive (Proton Drive). 56736 .SS --protondrive-mailbox-password 56737 .PP 56738 The mailbox password of your two-password proton account. 56739 .PP 56740 For more information regarding the mailbox password, please check the 56741 following official knowledge base article: 56742 https://proton.me/support/the-difference-between-the-mailbox-password-and-login-password 56743 .PP 56744 \f[B]NB\f[R] Input to this must be obscured - see rclone 56745 obscure (https://rclone.org/commands/rclone_obscure/). 56746 .PP 56747 Properties: 56748 .IP \[bu] 2 56749 Config: mailbox_password 56750 .IP \[bu] 2 56751 Env Var: RCLONE_PROTONDRIVE_MAILBOX_PASSWORD 56752 .IP \[bu] 2 56753 Type: string 56754 .IP \[bu] 2 56755 Required: false 56756 .SS --protondrive-client-uid 56757 .PP 56758 Client uid key (internal use only) 56759 .PP 56760 Properties: 56761 .IP \[bu] 2 56762 Config: client_uid 56763 .IP \[bu] 2 56764 Env Var: RCLONE_PROTONDRIVE_CLIENT_UID 56765 .IP \[bu] 2 56766 Type: string 56767 .IP \[bu] 2 56768 Required: false 56769 .SS --protondrive-client-access-token 56770 .PP 56771 Client access token key (internal use only) 56772 .PP 56773 Properties: 56774 .IP \[bu] 2 56775 Config: client_access_token 56776 .IP \[bu] 2 56777 Env Var: RCLONE_PROTONDRIVE_CLIENT_ACCESS_TOKEN 56778 .IP \[bu] 2 56779 Type: string 56780 .IP \[bu] 2 56781 Required: false 56782 .SS --protondrive-client-refresh-token 56783 .PP 56784 Client refresh token key (internal use only) 56785 .PP 56786 Properties: 56787 .IP \[bu] 2 56788 Config: client_refresh_token 56789 .IP \[bu] 2 56790 Env Var: RCLONE_PROTONDRIVE_CLIENT_REFRESH_TOKEN 56791 .IP \[bu] 2 56792 Type: string 56793 .IP \[bu] 2 56794 Required: false 56795 .SS --protondrive-client-salted-key-pass 56796 .PP 56797 Client salted key pass key (internal use only) 56798 .PP 56799 Properties: 56800 .IP \[bu] 2 56801 Config: client_salted_key_pass 56802 .IP \[bu] 2 56803 Env Var: RCLONE_PROTONDRIVE_CLIENT_SALTED_KEY_PASS 56804 .IP \[bu] 2 56805 Type: string 56806 .IP \[bu] 2 56807 Required: false 56808 .SS --protondrive-encoding 56809 .PP 56810 The encoding for the backend. 56811 .PP 56812 See the encoding section in the 56813 overview (https://rclone.org/overview/#encoding) for more info. 56814 .PP 56815 Properties: 56816 .IP \[bu] 2 56817 Config: encoding 56818 .IP \[bu] 2 56819 Env Var: RCLONE_PROTONDRIVE_ENCODING 56820 .IP \[bu] 2 56821 Type: Encoding 56822 .IP \[bu] 2 56823 Default: Slash,LeftSpace,RightSpace,InvalidUtf8,Dot 56824 .SS --protondrive-original-file-size 56825 .PP 56826 Return the file size before encryption 56827 .PP 56828 The size of the encrypted file will be different from (bigger than) the 56829 original file size. 56830 Unless there is a reason to return the file size after encryption is 56831 performed, otherwise, set this option to true, as features like Open() 56832 which will need to be supplied with original content size, will fail to 56833 operate properly 56834 .PP 56835 Properties: 56836 .IP \[bu] 2 56837 Config: original_file_size 56838 .IP \[bu] 2 56839 Env Var: RCLONE_PROTONDRIVE_ORIGINAL_FILE_SIZE 56840 .IP \[bu] 2 56841 Type: bool 56842 .IP \[bu] 2 56843 Default: true 56844 .SS --protondrive-app-version 56845 .PP 56846 The app version string 56847 .PP 56848 The app version string indicates the client that is currently performing 56849 the API request. 56850 This information is required and will be sent with every API request. 56851 .PP 56852 Properties: 56853 .IP \[bu] 2 56854 Config: app_version 56855 .IP \[bu] 2 56856 Env Var: RCLONE_PROTONDRIVE_APP_VERSION 56857 .IP \[bu] 2 56858 Type: string 56859 .IP \[bu] 2 56860 Default: \[dq]macos-drive\[at]1.0.0-alpha.1+rclone\[dq] 56861 .SS --protondrive-replace-existing-draft 56862 .PP 56863 Create a new revision when filename conflict is detected 56864 .PP 56865 When a file upload is cancelled or failed before completion, a draft 56866 will be created and the subsequent upload of the same file to the same 56867 location will be reported as a conflict. 56868 .PP 56869 The value can also be set by --protondrive-replace-existing-draft=true 56870 .PP 56871 If the option is set to true, the draft will be replaced and then the 56872 upload operation will restart. 56873 If there are other clients also uploading at the same file location at 56874 the same time, the behavior is currently unknown. 56875 Need to set to true for integration tests. 56876 If the option is set to false, an error \[dq]a draft exist - usually 56877 this means a file is being uploaded at another client, or, there was a 56878 failed upload attempt\[dq] will be returned, and no upload will happen. 56879 .PP 56880 Properties: 56881 .IP \[bu] 2 56882 Config: replace_existing_draft 56883 .IP \[bu] 2 56884 Env Var: RCLONE_PROTONDRIVE_REPLACE_EXISTING_DRAFT 56885 .IP \[bu] 2 56886 Type: bool 56887 .IP \[bu] 2 56888 Default: false 56889 .SS --protondrive-enable-caching 56890 .PP 56891 Caches the files and folders metadata to reduce API calls 56892 .PP 56893 Notice: If you are mounting ProtonDrive as a VFS, please disable this 56894 feature, as the current implementation doesn\[aq]t update or clear the 56895 cache when there are external changes. 56896 .PP 56897 The files and folders on ProtonDrive are represented as links with 56898 keyrings, which can be cached to improve performance and be friendly to 56899 the API server. 56900 .PP 56901 The cache is currently built for the case when the rclone is the only 56902 instance performing operations to the mount point. 56903 The event system, which is the proton API system that provides 56904 visibility of what has changed on the drive, is yet to be implemented, 56905 so updates from other clients won\[cq]t be reflected in the cache. 56906 Thus, if there are concurrent clients accessing the same mount point, 56907 then we might have a problem with caching the stale data. 56908 .PP 56909 Properties: 56910 .IP \[bu] 2 56911 Config: enable_caching 56912 .IP \[bu] 2 56913 Env Var: RCLONE_PROTONDRIVE_ENABLE_CACHING 56914 .IP \[bu] 2 56915 Type: bool 56916 .IP \[bu] 2 56917 Default: true 56918 .SS --protondrive-description 56919 .PP 56920 Description of the remote 56921 .PP 56922 Properties: 56923 .IP \[bu] 2 56924 Config: description 56925 .IP \[bu] 2 56926 Env Var: RCLONE_PROTONDRIVE_DESCRIPTION 56927 .IP \[bu] 2 56928 Type: string 56929 .IP \[bu] 2 56930 Required: false 56931 .SS Limitations 56932 .PP 56933 This backend uses the 56934 Proton-API-Bridge (https://github.com/henrybear327/Proton-API-Bridge), 56935 which is based on 56936 go-proton-api (https://github.com/henrybear327/go-proton-api), a fork of 56937 the official repo (https://github.com/ProtonMail/go-proton-api). 56938 .PP 56939 There is no official API documentation available from Proton Drive. 56940 But, thanks to Proton open sourcing 56941 proton-go-api (https://github.com/ProtonMail/go-proton-api) and the web, 56942 iOS, and Android client codebases, we don\[aq]t need to completely 56943 reverse engineer the APIs by observing the web client traffic! 56944 .PP 56945 proton-go-api (https://github.com/ProtonMail/go-proton-api) provides the 56946 basic building blocks of API calls and error handling, such as 429 56947 exponential back-off, but it is pretty much just a barebone interface to 56948 the Proton API. 56949 For example, the encryption and decryption of the Proton Drive file are 56950 not provided in this library. 56951 .PP 56952 The Proton-API-Bridge, attempts to bridge the gap, so rclone can be 56953 built on top of this quickly. 56954 This codebase handles the intricate tasks before and after calling 56955 Proton APIs, particularly the complex encryption scheme, allowing 56956 developers to implement features for other software on top of this 56957 codebase. 56958 There are likely quite a few errors in this library, as there isn\[aq]t 56959 official documentation available. 56960 .SH put.io 56961 .PP 56962 Paths are specified as \f[C]remote:path\f[R] 56963 .PP 56964 put.io paths may be as deep as required, e.g. 56965 \f[C]remote:directory/subdirectory\f[R]. 56966 .SS Configuration 56967 .PP 56968 The initial setup for put.io involves getting a token from put.io which 56969 you need to do in your browser. 56970 \f[C]rclone config\f[R] walks you through it. 56971 .PP 56972 Here is an example of how to make a remote called \f[C]remote\f[R]. 56973 First run: 56974 .IP 56975 .nf 56976 \f[C] 56977 rclone config 56978 \f[R] 56979 .fi 56980 .PP 56981 This will guide you through an interactive setup process: 56982 .IP 56983 .nf 56984 \f[C] 56985 No remotes found, make a new one? 56986 n) New remote 56987 s) Set configuration password 56988 q) Quit config 56989 n/s/q> n 56990 name> putio 56991 Type of storage to configure. 56992 Enter a string value. Press Enter for the default (\[dq]\[dq]). 56993 Choose a number from below, or type in your own value 56994 [snip] 56995 XX / Put.io 56996 \[rs] \[dq]putio\[dq] 56997 [snip] 56998 Storage> putio 56999 ** See help for putio backend at: https://rclone.org/putio/ ** 57000 57001 Remote config 57002 Use web browser to automatically authenticate rclone with remote? 57003 * Say Y if the machine running rclone has a web browser you can use 57004 * Say N if running rclone on a (remote) machine without web browser access 57005 If not sure try Y. If Y failed, try N. 57006 y) Yes 57007 n) No 57008 y/n> y 57009 If your browser doesn\[aq]t open automatically go to the following link: http://127.0.0.1:53682/auth 57010 Log in and authorize rclone for access 57011 Waiting for code... 57012 Got code 57013 -------------------- 57014 [putio] 57015 type = putio 57016 token = {\[dq]access_token\[dq]:\[dq]XXXXXXXX\[dq],\[dq]expiry\[dq]:\[dq]0001-01-01T00:00:00Z\[dq]} 57017 -------------------- 57018 y) Yes this is OK 57019 e) Edit this remote 57020 d) Delete this remote 57021 y/e/d> y 57022 Current remotes: 57023 57024 Name Type 57025 ==== ==== 57026 putio putio 57027 57028 e) Edit existing remote 57029 n) New remote 57030 d) Delete remote 57031 r) Rename remote 57032 c) Copy remote 57033 s) Set configuration password 57034 q) Quit config 57035 e/n/d/r/c/s/q> q 57036 \f[R] 57037 .fi 57038 .PP 57039 See the remote setup docs (https://rclone.org/remote_setup/) for how to 57040 set it up on a machine with no Internet browser available. 57041 .PP 57042 Note that rclone runs a webserver on your local machine to collect the 57043 token as returned from put.io if using web browser to automatically 57044 authenticate. 57045 This only runs from the moment it opens your browser to the moment you 57046 get back the verification code. 57047 This is on \f[C]http://127.0.0.1:53682/\f[R] and this it may require you 57048 to unblock it temporarily if you are running a host firewall, or use 57049 manual mode. 57050 .PP 57051 You can then use it like this, 57052 .PP 57053 List directories in top level of your put.io 57054 .IP 57055 .nf 57056 \f[C] 57057 rclone lsd remote: 57058 \f[R] 57059 .fi 57060 .PP 57061 List all the files in your put.io 57062 .IP 57063 .nf 57064 \f[C] 57065 rclone ls remote: 57066 \f[R] 57067 .fi 57068 .PP 57069 To copy a local directory to a put.io directory called backup 57070 .IP 57071 .nf 57072 \f[C] 57073 rclone copy /home/source remote:backup 57074 \f[R] 57075 .fi 57076 .SS Restricted filename characters 57077 .PP 57078 In addition to the default restricted characters 57079 set (https://rclone.org/overview/#restricted-characters) the following 57080 characters are also replaced: 57081 .PP 57082 .TS 57083 tab(@); 57084 l c c. 57085 T{ 57086 Character 57087 T}@T{ 57088 Value 57089 T}@T{ 57090 Replacement 57091 T} 57092 _ 57093 T{ 57094 \[rs] 57095 T}@T{ 57096 0x5C 57097 T}@T{ 57098 \[uFF3C] 57099 T} 57100 .TE 57101 .PP 57102 Invalid UTF-8 bytes will also be 57103 replaced (https://rclone.org/overview/#invalid-utf8), as they can\[aq]t 57104 be used in JSON strings. 57105 .SS Standard options 57106 .PP 57107 Here are the Standard options specific to putio (Put.io). 57108 .SS --putio-client-id 57109 .PP 57110 OAuth Client Id. 57111 .PP 57112 Leave blank normally. 57113 .PP 57114 Properties: 57115 .IP \[bu] 2 57116 Config: client_id 57117 .IP \[bu] 2 57118 Env Var: RCLONE_PUTIO_CLIENT_ID 57119 .IP \[bu] 2 57120 Type: string 57121 .IP \[bu] 2 57122 Required: false 57123 .SS --putio-client-secret 57124 .PP 57125 OAuth Client Secret. 57126 .PP 57127 Leave blank normally. 57128 .PP 57129 Properties: 57130 .IP \[bu] 2 57131 Config: client_secret 57132 .IP \[bu] 2 57133 Env Var: RCLONE_PUTIO_CLIENT_SECRET 57134 .IP \[bu] 2 57135 Type: string 57136 .IP \[bu] 2 57137 Required: false 57138 .SS Advanced options 57139 .PP 57140 Here are the Advanced options specific to putio (Put.io). 57141 .SS --putio-token 57142 .PP 57143 OAuth Access Token as a JSON blob. 57144 .PP 57145 Properties: 57146 .IP \[bu] 2 57147 Config: token 57148 .IP \[bu] 2 57149 Env Var: RCLONE_PUTIO_TOKEN 57150 .IP \[bu] 2 57151 Type: string 57152 .IP \[bu] 2 57153 Required: false 57154 .SS --putio-auth-url 57155 .PP 57156 Auth server URL. 57157 .PP 57158 Leave blank to use the provider defaults. 57159 .PP 57160 Properties: 57161 .IP \[bu] 2 57162 Config: auth_url 57163 .IP \[bu] 2 57164 Env Var: RCLONE_PUTIO_AUTH_URL 57165 .IP \[bu] 2 57166 Type: string 57167 .IP \[bu] 2 57168 Required: false 57169 .SS --putio-token-url 57170 .PP 57171 Token server url. 57172 .PP 57173 Leave blank to use the provider defaults. 57174 .PP 57175 Properties: 57176 .IP \[bu] 2 57177 Config: token_url 57178 .IP \[bu] 2 57179 Env Var: RCLONE_PUTIO_TOKEN_URL 57180 .IP \[bu] 2 57181 Type: string 57182 .IP \[bu] 2 57183 Required: false 57184 .SS --putio-encoding 57185 .PP 57186 The encoding for the backend. 57187 .PP 57188 See the encoding section in the 57189 overview (https://rclone.org/overview/#encoding) for more info. 57190 .PP 57191 Properties: 57192 .IP \[bu] 2 57193 Config: encoding 57194 .IP \[bu] 2 57195 Env Var: RCLONE_PUTIO_ENCODING 57196 .IP \[bu] 2 57197 Type: Encoding 57198 .IP \[bu] 2 57199 Default: Slash,BackSlash,Del,Ctl,InvalidUtf8,Dot 57200 .SS --putio-description 57201 .PP 57202 Description of the remote 57203 .PP 57204 Properties: 57205 .IP \[bu] 2 57206 Config: description 57207 .IP \[bu] 2 57208 Env Var: RCLONE_PUTIO_DESCRIPTION 57209 .IP \[bu] 2 57210 Type: string 57211 .IP \[bu] 2 57212 Required: false 57213 .SS Limitations 57214 .PP 57215 put.io has rate limiting. 57216 When you hit a limit, rclone automatically retries after waiting the 57217 amount of time requested by the server. 57218 .PP 57219 If you want to avoid ever hitting these limits, you may use the 57220 \f[C]--tpslimit\f[R] flag with a low number. 57221 Note that the imposed limits may be different for different operations, 57222 and may change over time. 57223 .SH Proton Drive 57224 .PP 57225 Proton Drive (https://proton.me/drive) is an end-to-end encrypted Swiss 57226 vault for your files that protects your data. 57227 .PP 57228 This is an rclone backend for Proton Drive which supports the file 57229 transfer features of Proton Drive using the same client-side encryption. 57230 .PP 57231 Due to the fact that Proton Drive doesn\[aq]t publish its API 57232 documentation, this backend is implemented with best efforts by reading 57233 the open-sourced client source code and observing the Proton Drive 57234 traffic in the browser. 57235 .PP 57236 \f[B]NB\f[R] This backend is currently in Beta. 57237 It is believed to be correct and all the integration tests pass. 57238 However the Proton Drive protocol has evolved over time there may be 57239 accounts it is not compatible with. 57240 Please post on the rclone forum (https://forum.rclone.org/) if you find 57241 an incompatibility. 57242 .PP 57243 Paths are specified as \f[C]remote:path\f[R] 57244 .PP 57245 Paths may be as deep as required, e.g. 57246 \f[C]remote:directory/subdirectory\f[R]. 57247 .SS Configurations 57248 .PP 57249 Here is an example of how to make a remote called \f[C]remote\f[R]. 57250 First run: 57251 .IP 57252 .nf 57253 \f[C] 57254 rclone config 57255 \f[R] 57256 .fi 57257 .PP 57258 This will guide you through an interactive setup process: 57259 .IP 57260 .nf 57261 \f[C] 57262 No remotes found, make a new one? 57263 n) New remote 57264 s) Set configuration password 57265 q) Quit config 57266 n/s/q> n 57267 name> remote 57268 Type of storage to configure. 57269 Choose a number from below, or type in your own value 57270 [snip] 57271 XX / Proton Drive 57272 \[rs] \[dq]Proton Drive\[dq] 57273 [snip] 57274 Storage> protondrive 57275 User name 57276 user> you\[at]protonmail.com 57277 Password. 57278 y) Yes type in my own password 57279 g) Generate random password 57280 n) No leave this optional password blank 57281 y/g/n> y 57282 Enter the password: 57283 password: 57284 Confirm the password: 57285 password: 57286 Option 2fa. 57287 2FA code (if the account requires one) 57288 Enter a value. Press Enter to leave empty. 57289 2fa> 123456 57290 Remote config 57291 -------------------- 57292 [remote] 57293 type = protondrive 57294 user = you\[at]protonmail.com 57295 pass = *** ENCRYPTED *** 57296 -------------------- 57297 y) Yes this is OK 57298 e) Edit this remote 57299 d) Delete this remote 57300 y/e/d> y 57301 \f[R] 57302 .fi 57303 .PP 57304 \f[B]NOTE:\f[R] The Proton Drive encryption keys need to have been 57305 already generated after a regular login via the browser, otherwise 57306 attempting to use the credentials in \f[C]rclone\f[R] will fail. 57307 .PP 57308 Once configured you can then use \f[C]rclone\f[R] like this, 57309 .PP 57310 List directories in top level of your Proton Drive 57311 .IP 57312 .nf 57313 \f[C] 57314 rclone lsd remote: 57315 \f[R] 57316 .fi 57317 .PP 57318 List all the files in your Proton Drive 57319 .IP 57320 .nf 57321 \f[C] 57322 rclone ls remote: 57323 \f[R] 57324 .fi 57325 .PP 57326 To copy a local directory to an Proton Drive directory called backup 57327 .IP 57328 .nf 57329 \f[C] 57330 rclone copy /home/source remote:backup 57331 \f[R] 57332 .fi 57333 .SS Modification times and hashes 57334 .PP 57335 Proton Drive Bridge does not support updating modification times yet. 57336 .PP 57337 The SHA1 hash algorithm is supported. 57338 .SS Restricted filename characters 57339 .PP 57340 Invalid UTF-8 bytes will be 57341 replaced (https://rclone.org/overview/#invalid-utf8), also left and 57342 right spaces will be removed (code 57343 reference (https://github.com/ProtonMail/WebClients/blob/b4eba99d241af4fdae06ff7138bd651a40ef5d3c/applications/drive/src/app/store/_links/validation.ts#L51)) 57344 .SS Duplicated files 57345 .PP 57346 Proton Drive can not have two files with exactly the same name and path. 57347 If the conflict occurs, depending on the advanced config, the file might 57348 or might not be overwritten. 57349 .SS Mailbox password (https://proton.me/support/the-difference-between-the-mailbox-password-and-login-password) 57350 .PP 57351 Please set your mailbox password in the advanced config section. 57352 .SS Caching 57353 .PP 57354 The cache is currently built for the case when the rclone is the only 57355 instance performing operations to the mount point. 57356 The event system, which is the proton API system that provides 57357 visibility of what has changed on the drive, is yet to be implemented, 57358 so updates from other clients won\[cq]t be reflected in the cache. 57359 Thus, if there are concurrent clients accessing the same mount point, 57360 then we might have a problem with caching the stale data. 57361 .SS Standard options 57362 .PP 57363 Here are the Standard options specific to protondrive (Proton Drive). 57364 .SS --protondrive-username 57365 .PP 57366 The username of your proton account 57367 .PP 57368 Properties: 57369 .IP \[bu] 2 57370 Config: username 57371 .IP \[bu] 2 57372 Env Var: RCLONE_PROTONDRIVE_USERNAME 57373 .IP \[bu] 2 57374 Type: string 57375 .IP \[bu] 2 57376 Required: true 57377 .SS --protondrive-password 57378 .PP 57379 The password of your proton account. 57380 .PP 57381 \f[B]NB\f[R] Input to this must be obscured - see rclone 57382 obscure (https://rclone.org/commands/rclone_obscure/). 57383 .PP 57384 Properties: 57385 .IP \[bu] 2 57386 Config: password 57387 .IP \[bu] 2 57388 Env Var: RCLONE_PROTONDRIVE_PASSWORD 57389 .IP \[bu] 2 57390 Type: string 57391 .IP \[bu] 2 57392 Required: true 57393 .SS --protondrive-2fa 57394 .PP 57395 The 2FA code 57396 .PP 57397 The value can also be provided with --protondrive-2fa=000000 57398 .PP 57399 The 2FA code of your proton drive account if the account is set up with 57400 two-factor authentication 57401 .PP 57402 Properties: 57403 .IP \[bu] 2 57404 Config: 2fa 57405 .IP \[bu] 2 57406 Env Var: RCLONE_PROTONDRIVE_2FA 57407 .IP \[bu] 2 57408 Type: string 57409 .IP \[bu] 2 57410 Required: false 57411 .SS Advanced options 57412 .PP 57413 Here are the Advanced options specific to protondrive (Proton Drive). 57414 .SS --protondrive-mailbox-password 57415 .PP 57416 The mailbox password of your two-password proton account. 57417 .PP 57418 For more information regarding the mailbox password, please check the 57419 following official knowledge base article: 57420 https://proton.me/support/the-difference-between-the-mailbox-password-and-login-password 57421 .PP 57422 \f[B]NB\f[R] Input to this must be obscured - see rclone 57423 obscure (https://rclone.org/commands/rclone_obscure/). 57424 .PP 57425 Properties: 57426 .IP \[bu] 2 57427 Config: mailbox_password 57428 .IP \[bu] 2 57429 Env Var: RCLONE_PROTONDRIVE_MAILBOX_PASSWORD 57430 .IP \[bu] 2 57431 Type: string 57432 .IP \[bu] 2 57433 Required: false 57434 .SS --protondrive-client-uid 57435 .PP 57436 Client uid key (internal use only) 57437 .PP 57438 Properties: 57439 .IP \[bu] 2 57440 Config: client_uid 57441 .IP \[bu] 2 57442 Env Var: RCLONE_PROTONDRIVE_CLIENT_UID 57443 .IP \[bu] 2 57444 Type: string 57445 .IP \[bu] 2 57446 Required: false 57447 .SS --protondrive-client-access-token 57448 .PP 57449 Client access token key (internal use only) 57450 .PP 57451 Properties: 57452 .IP \[bu] 2 57453 Config: client_access_token 57454 .IP \[bu] 2 57455 Env Var: RCLONE_PROTONDRIVE_CLIENT_ACCESS_TOKEN 57456 .IP \[bu] 2 57457 Type: string 57458 .IP \[bu] 2 57459 Required: false 57460 .SS --protondrive-client-refresh-token 57461 .PP 57462 Client refresh token key (internal use only) 57463 .PP 57464 Properties: 57465 .IP \[bu] 2 57466 Config: client_refresh_token 57467 .IP \[bu] 2 57468 Env Var: RCLONE_PROTONDRIVE_CLIENT_REFRESH_TOKEN 57469 .IP \[bu] 2 57470 Type: string 57471 .IP \[bu] 2 57472 Required: false 57473 .SS --protondrive-client-salted-key-pass 57474 .PP 57475 Client salted key pass key (internal use only) 57476 .PP 57477 Properties: 57478 .IP \[bu] 2 57479 Config: client_salted_key_pass 57480 .IP \[bu] 2 57481 Env Var: RCLONE_PROTONDRIVE_CLIENT_SALTED_KEY_PASS 57482 .IP \[bu] 2 57483 Type: string 57484 .IP \[bu] 2 57485 Required: false 57486 .SS --protondrive-encoding 57487 .PP 57488 The encoding for the backend. 57489 .PP 57490 See the encoding section in the 57491 overview (https://rclone.org/overview/#encoding) for more info. 57492 .PP 57493 Properties: 57494 .IP \[bu] 2 57495 Config: encoding 57496 .IP \[bu] 2 57497 Env Var: RCLONE_PROTONDRIVE_ENCODING 57498 .IP \[bu] 2 57499 Type: Encoding 57500 .IP \[bu] 2 57501 Default: Slash,LeftSpace,RightSpace,InvalidUtf8,Dot 57502 .SS --protondrive-original-file-size 57503 .PP 57504 Return the file size before encryption 57505 .PP 57506 The size of the encrypted file will be different from (bigger than) the 57507 original file size. 57508 Unless there is a reason to return the file size after encryption is 57509 performed, otherwise, set this option to true, as features like Open() 57510 which will need to be supplied with original content size, will fail to 57511 operate properly 57512 .PP 57513 Properties: 57514 .IP \[bu] 2 57515 Config: original_file_size 57516 .IP \[bu] 2 57517 Env Var: RCLONE_PROTONDRIVE_ORIGINAL_FILE_SIZE 57518 .IP \[bu] 2 57519 Type: bool 57520 .IP \[bu] 2 57521 Default: true 57522 .SS --protondrive-app-version 57523 .PP 57524 The app version string 57525 .PP 57526 The app version string indicates the client that is currently performing 57527 the API request. 57528 This information is required and will be sent with every API request. 57529 .PP 57530 Properties: 57531 .IP \[bu] 2 57532 Config: app_version 57533 .IP \[bu] 2 57534 Env Var: RCLONE_PROTONDRIVE_APP_VERSION 57535 .IP \[bu] 2 57536 Type: string 57537 .IP \[bu] 2 57538 Default: \[dq]macos-drive\[at]1.0.0-alpha.1+rclone\[dq] 57539 .SS --protondrive-replace-existing-draft 57540 .PP 57541 Create a new revision when filename conflict is detected 57542 .PP 57543 When a file upload is cancelled or failed before completion, a draft 57544 will be created and the subsequent upload of the same file to the same 57545 location will be reported as a conflict. 57546 .PP 57547 The value can also be set by --protondrive-replace-existing-draft=true 57548 .PP 57549 If the option is set to true, the draft will be replaced and then the 57550 upload operation will restart. 57551 If there are other clients also uploading at the same file location at 57552 the same time, the behavior is currently unknown. 57553 Need to set to true for integration tests. 57554 If the option is set to false, an error \[dq]a draft exist - usually 57555 this means a file is being uploaded at another client, or, there was a 57556 failed upload attempt\[dq] will be returned, and no upload will happen. 57557 .PP 57558 Properties: 57559 .IP \[bu] 2 57560 Config: replace_existing_draft 57561 .IP \[bu] 2 57562 Env Var: RCLONE_PROTONDRIVE_REPLACE_EXISTING_DRAFT 57563 .IP \[bu] 2 57564 Type: bool 57565 .IP \[bu] 2 57566 Default: false 57567 .SS --protondrive-enable-caching 57568 .PP 57569 Caches the files and folders metadata to reduce API calls 57570 .PP 57571 Notice: If you are mounting ProtonDrive as a VFS, please disable this 57572 feature, as the current implementation doesn\[aq]t update or clear the 57573 cache when there are external changes. 57574 .PP 57575 The files and folders on ProtonDrive are represented as links with 57576 keyrings, which can be cached to improve performance and be friendly to 57577 the API server. 57578 .PP 57579 The cache is currently built for the case when the rclone is the only 57580 instance performing operations to the mount point. 57581 The event system, which is the proton API system that provides 57582 visibility of what has changed on the drive, is yet to be implemented, 57583 so updates from other clients won\[cq]t be reflected in the cache. 57584 Thus, if there are concurrent clients accessing the same mount point, 57585 then we might have a problem with caching the stale data. 57586 .PP 57587 Properties: 57588 .IP \[bu] 2 57589 Config: enable_caching 57590 .IP \[bu] 2 57591 Env Var: RCLONE_PROTONDRIVE_ENABLE_CACHING 57592 .IP \[bu] 2 57593 Type: bool 57594 .IP \[bu] 2 57595 Default: true 57596 .SS --protondrive-description 57597 .PP 57598 Description of the remote 57599 .PP 57600 Properties: 57601 .IP \[bu] 2 57602 Config: description 57603 .IP \[bu] 2 57604 Env Var: RCLONE_PROTONDRIVE_DESCRIPTION 57605 .IP \[bu] 2 57606 Type: string 57607 .IP \[bu] 2 57608 Required: false 57609 .SS Limitations 57610 .PP 57611 This backend uses the 57612 Proton-API-Bridge (https://github.com/henrybear327/Proton-API-Bridge), 57613 which is based on 57614 go-proton-api (https://github.com/henrybear327/go-proton-api), a fork of 57615 the official repo (https://github.com/ProtonMail/go-proton-api). 57616 .PP 57617 There is no official API documentation available from Proton Drive. 57618 But, thanks to Proton open sourcing 57619 proton-go-api (https://github.com/ProtonMail/go-proton-api) and the web, 57620 iOS, and Android client codebases, we don\[aq]t need to completely 57621 reverse engineer the APIs by observing the web client traffic! 57622 .PP 57623 proton-go-api (https://github.com/ProtonMail/go-proton-api) provides the 57624 basic building blocks of API calls and error handling, such as 429 57625 exponential back-off, but it is pretty much just a barebone interface to 57626 the Proton API. 57627 For example, the encryption and decryption of the Proton Drive file are 57628 not provided in this library. 57629 .PP 57630 The Proton-API-Bridge, attempts to bridge the gap, so rclone can be 57631 built on top of this quickly. 57632 This codebase handles the intricate tasks before and after calling 57633 Proton APIs, particularly the complex encryption scheme, allowing 57634 developers to implement features for other software on top of this 57635 codebase. 57636 There are likely quite a few errors in this library, as there isn\[aq]t 57637 official documentation available. 57638 .SH Seafile 57639 .PP 57640 This is a backend for the Seafile (https://www.seafile.com/) storage 57641 service: - It works with both the free community edition or the 57642 professional edition. 57643 - Seafile versions 6.x, 7.x, 8.x and 9.x are all supported. 57644 - Encrypted libraries are also supported. 57645 - It supports 2FA enabled users - Using a Library API Token is 57646 \f[B]not\f[R] supported 57647 .SS Configuration 57648 .PP 57649 There are two distinct modes you can setup your remote: - you point your 57650 remote to the \f[B]root of the server\f[R], meaning you don\[aq]t 57651 specify a library during the configuration: Paths are specified as 57652 \f[C]remote:library\f[R]. 57653 You may put subdirectories in too, e.g. 57654 \f[C]remote:library/path/to/dir\f[R]. 57655 - you point your remote to a specific library during the configuration: 57656 Paths are specified as \f[C]remote:path/to/dir\f[R]. 57657 \f[B]This is the recommended mode when using encrypted libraries\f[R]. 57658 (\f[I]This mode is possibly slightly faster than the root mode\f[R]) 57659 .SS Configuration in root mode 57660 .PP 57661 Here is an example of making a seafile configuration for a user with 57662 \f[B]no\f[R] two-factor authentication. 57663 First run 57664 .IP 57665 .nf 57666 \f[C] 57667 rclone config 57668 \f[R] 57669 .fi 57670 .PP 57671 This will guide you through an interactive setup process. 57672 To authenticate you will need the URL of your server, your email (or 57673 username) and your password. 57674 .IP 57675 .nf 57676 \f[C] 57677 No remotes found, make a new one? 57678 n) New remote 57679 s) Set configuration password 57680 q) Quit config 57681 n/s/q> n 57682 name> seafile 57683 Type of storage to configure. 57684 Enter a string value. Press Enter for the default (\[dq]\[dq]). 57685 Choose a number from below, or type in your own value 57686 [snip] 57687 XX / Seafile 57688 \[rs] \[dq]seafile\[dq] 57689 [snip] 57690 Storage> seafile 57691 ** See help for seafile backend at: https://rclone.org/seafile/ ** 57692 57693 URL of seafile host to connect to 57694 Enter a string value. Press Enter for the default (\[dq]\[dq]). 57695 Choose a number from below, or type in your own value 57696 1 / Connect to cloud.seafile.com 57697 \[rs] \[dq]https://cloud.seafile.com/\[dq] 57698 url> http://my.seafile.server/ 57699 User name (usually email address) 57700 Enter a string value. Press Enter for the default (\[dq]\[dq]). 57701 user> me\[at]example.com 57702 Password 57703 y) Yes type in my own password 57704 g) Generate random password 57705 n) No leave this optional password blank (default) 57706 y/g> y 57707 Enter the password: 57708 password: 57709 Confirm the password: 57710 password: 57711 Two-factor authentication (\[aq]true\[aq] if the account has 2FA enabled) 57712 Enter a boolean value (true or false). Press Enter for the default (\[dq]false\[dq]). 57713 2fa> false 57714 Name of the library. Leave blank to access all non-encrypted libraries. 57715 Enter a string value. Press Enter for the default (\[dq]\[dq]). 57716 library> 57717 Library password (for encrypted libraries only). Leave blank if you pass it through the command line. 57718 y) Yes type in my own password 57719 g) Generate random password 57720 n) No leave this optional password blank (default) 57721 y/g/n> n 57722 Edit advanced config? (y/n) 57723 y) Yes 57724 n) No (default) 57725 y/n> n 57726 Remote config 57727 Two-factor authentication is not enabled on this account. 57728 -------------------- 57729 [seafile] 57730 type = seafile 57731 url = http://my.seafile.server/ 57732 user = me\[at]example.com 57733 pass = *** ENCRYPTED *** 57734 2fa = false 57735 -------------------- 57736 y) Yes this is OK (default) 57737 e) Edit this remote 57738 d) Delete this remote 57739 y/e/d> y 57740 \f[R] 57741 .fi 57742 .PP 57743 This remote is called \f[C]seafile\f[R]. 57744 It\[aq]s pointing to the root of your seafile server and can now be used 57745 like this: 57746 .PP 57747 See all libraries 57748 .IP 57749 .nf 57750 \f[C] 57751 rclone lsd seafile: 57752 \f[R] 57753 .fi 57754 .PP 57755 Create a new library 57756 .IP 57757 .nf 57758 \f[C] 57759 rclone mkdir seafile:library 57760 \f[R] 57761 .fi 57762 .PP 57763 List the contents of a library 57764 .IP 57765 .nf 57766 \f[C] 57767 rclone ls seafile:library 57768 \f[R] 57769 .fi 57770 .PP 57771 Sync \f[C]/home/local/directory\f[R] to the remote library, deleting any 57772 excess files in the library. 57773 .IP 57774 .nf 57775 \f[C] 57776 rclone sync --interactive /home/local/directory seafile:library 57777 \f[R] 57778 .fi 57779 .SS Configuration in library mode 57780 .PP 57781 Here\[aq]s an example of a configuration in library mode with a user 57782 that has the two-factor authentication enabled. 57783 Your 2FA code will be asked at the end of the configuration, and will 57784 attempt to authenticate you: 57785 .IP 57786 .nf 57787 \f[C] 57788 No remotes found, make a new one? 57789 n) New remote 57790 s) Set configuration password 57791 q) Quit config 57792 n/s/q> n 57793 name> seafile 57794 Type of storage to configure. 57795 Enter a string value. Press Enter for the default (\[dq]\[dq]). 57796 Choose a number from below, or type in your own value 57797 [snip] 57798 XX / Seafile 57799 \[rs] \[dq]seafile\[dq] 57800 [snip] 57801 Storage> seafile 57802 ** See help for seafile backend at: https://rclone.org/seafile/ ** 57803 57804 URL of seafile host to connect to 57805 Enter a string value. Press Enter for the default (\[dq]\[dq]). 57806 Choose a number from below, or type in your own value 57807 1 / Connect to cloud.seafile.com 57808 \[rs] \[dq]https://cloud.seafile.com/\[dq] 57809 url> http://my.seafile.server/ 57810 User name (usually email address) 57811 Enter a string value. Press Enter for the default (\[dq]\[dq]). 57812 user> me\[at]example.com 57813 Password 57814 y) Yes type in my own password 57815 g) Generate random password 57816 n) No leave this optional password blank (default) 57817 y/g> y 57818 Enter the password: 57819 password: 57820 Confirm the password: 57821 password: 57822 Two-factor authentication (\[aq]true\[aq] if the account has 2FA enabled) 57823 Enter a boolean value (true or false). Press Enter for the default (\[dq]false\[dq]). 57824 2fa> true 57825 Name of the library. Leave blank to access all non-encrypted libraries. 57826 Enter a string value. Press Enter for the default (\[dq]\[dq]). 57827 library> My Library 57828 Library password (for encrypted libraries only). Leave blank if you pass it through the command line. 57829 y) Yes type in my own password 57830 g) Generate random password 57831 n) No leave this optional password blank (default) 57832 y/g/n> n 57833 Edit advanced config? (y/n) 57834 y) Yes 57835 n) No (default) 57836 y/n> n 57837 Remote config 57838 Two-factor authentication: please enter your 2FA code 57839 2fa code> 123456 57840 Authenticating... 57841 Success! 57842 -------------------- 57843 [seafile] 57844 type = seafile 57845 url = http://my.seafile.server/ 57846 user = me\[at]example.com 57847 pass = 57848 2fa = true 57849 library = My Library 57850 -------------------- 57851 y) Yes this is OK (default) 57852 e) Edit this remote 57853 d) Delete this remote 57854 y/e/d> y 57855 \f[R] 57856 .fi 57857 .PP 57858 You\[aq]ll notice your password is blank in the configuration. 57859 It\[aq]s because we only need the password to authenticate you once. 57860 .PP 57861 You specified \f[C]My Library\f[R] during the configuration. 57862 The root of the remote is pointing at the root of the library 57863 \f[C]My Library\f[R]: 57864 .PP 57865 See all files in the library: 57866 .IP 57867 .nf 57868 \f[C] 57869 rclone lsd seafile: 57870 \f[R] 57871 .fi 57872 .PP 57873 Create a new directory inside the library 57874 .IP 57875 .nf 57876 \f[C] 57877 rclone mkdir seafile:directory 57878 \f[R] 57879 .fi 57880 .PP 57881 List the contents of a directory 57882 .IP 57883 .nf 57884 \f[C] 57885 rclone ls seafile:directory 57886 \f[R] 57887 .fi 57888 .PP 57889 Sync \f[C]/home/local/directory\f[R] to the remote library, deleting any 57890 excess files in the library. 57891 .IP 57892 .nf 57893 \f[C] 57894 rclone sync --interactive /home/local/directory seafile: 57895 \f[R] 57896 .fi 57897 .SS --fast-list 57898 .PP 57899 Seafile version 7+ supports \f[C]--fast-list\f[R] which allows you to 57900 use fewer transactions in exchange for more memory. 57901 See the rclone docs (https://rclone.org/docs/#fast-list) for more 57902 details. 57903 Please note this is not supported on seafile server version 6.x 57904 .SS Restricted filename characters 57905 .PP 57906 In addition to the default restricted characters 57907 set (https://rclone.org/overview/#restricted-characters) the following 57908 characters are also replaced: 57909 .PP 57910 .TS 57911 tab(@); 57912 l c c. 57913 T{ 57914 Character 57915 T}@T{ 57916 Value 57917 T}@T{ 57918 Replacement 57919 T} 57920 _ 57921 T{ 57922 / 57923 T}@T{ 57924 0x2F 57925 T}@T{ 57926 \[uFF0F] 57927 T} 57928 T{ 57929 \[dq] 57930 T}@T{ 57931 0x22 57932 T}@T{ 57933 \[uFF02] 57934 T} 57935 T{ 57936 \[rs] 57937 T}@T{ 57938 0x5C 57939 T}@T{ 57940 \[uFF3C] 57941 T} 57942 .TE 57943 .PP 57944 Invalid UTF-8 bytes will also be 57945 replaced (https://rclone.org/overview/#invalid-utf8), as they can\[aq]t 57946 be used in JSON strings. 57947 .SS Seafile and rclone link 57948 .PP 57949 Rclone supports generating share links for non-encrypted libraries only. 57950 They can either be for a file or a directory: 57951 .IP 57952 .nf 57953 \f[C] 57954 rclone link seafile:seafile-tutorial.doc 57955 http://my.seafile.server/f/fdcd8a2f93f84b8b90f4/ 57956 \f[R] 57957 .fi 57958 .PP 57959 or if run on a directory you will get: 57960 .IP 57961 .nf 57962 \f[C] 57963 rclone link seafile:dir 57964 http://my.seafile.server/d/9ea2455f6f55478bbb0d/ 57965 \f[R] 57966 .fi 57967 .PP 57968 Please note a share link is unique for each file or directory. 57969 If you run a link command on a file/dir that has already been shared, 57970 you will get the exact same link. 57971 .SS Compatibility 57972 .PP 57973 It has been actively developed using the seafile docker 57974 image (https://github.com/haiwen/seafile-docker) of these versions: - 57975 6.3.4 community edition - 7.0.5 community edition - 7.1.3 community 57976 edition - 9.0.10 community edition 57977 .PP 57978 Versions below 6.0 are not supported. 57979 Versions between 6.0 and 6.3 haven\[aq]t been tested and might not work 57980 properly. 57981 .PP 57982 Each new version of \f[C]rclone\f[R] is automatically tested against the 57983 latest docker image (https://hub.docker.com/r/seafileltd/seafile-mc/) of 57984 the seafile community server. 57985 .SS Standard options 57986 .PP 57987 Here are the Standard options specific to seafile (seafile). 57988 .SS --seafile-url 57989 .PP 57990 URL of seafile host to connect to. 57991 .PP 57992 Properties: 57993 .IP \[bu] 2 57994 Config: url 57995 .IP \[bu] 2 57996 Env Var: RCLONE_SEAFILE_URL 57997 .IP \[bu] 2 57998 Type: string 57999 .IP \[bu] 2 58000 Required: true 58001 .IP \[bu] 2 58002 Examples: 58003 .RS 2 58004 .IP \[bu] 2 58005 \[dq]https://cloud.seafile.com/\[dq] 58006 .RS 2 58007 .IP \[bu] 2 58008 Connect to cloud.seafile.com. 58009 .RE 58010 .RE 58011 .SS --seafile-user 58012 .PP 58013 User name (usually email address). 58014 .PP 58015 Properties: 58016 .IP \[bu] 2 58017 Config: user 58018 .IP \[bu] 2 58019 Env Var: RCLONE_SEAFILE_USER 58020 .IP \[bu] 2 58021 Type: string 58022 .IP \[bu] 2 58023 Required: true 58024 .SS --seafile-pass 58025 .PP 58026 Password. 58027 .PP 58028 \f[B]NB\f[R] Input to this must be obscured - see rclone 58029 obscure (https://rclone.org/commands/rclone_obscure/). 58030 .PP 58031 Properties: 58032 .IP \[bu] 2 58033 Config: pass 58034 .IP \[bu] 2 58035 Env Var: RCLONE_SEAFILE_PASS 58036 .IP \[bu] 2 58037 Type: string 58038 .IP \[bu] 2 58039 Required: false 58040 .SS --seafile-2fa 58041 .PP 58042 Two-factor authentication (\[aq]true\[aq] if the account has 2FA 58043 enabled). 58044 .PP 58045 Properties: 58046 .IP \[bu] 2 58047 Config: 2fa 58048 .IP \[bu] 2 58049 Env Var: RCLONE_SEAFILE_2FA 58050 .IP \[bu] 2 58051 Type: bool 58052 .IP \[bu] 2 58053 Default: false 58054 .SS --seafile-library 58055 .PP 58056 Name of the library. 58057 .PP 58058 Leave blank to access all non-encrypted libraries. 58059 .PP 58060 Properties: 58061 .IP \[bu] 2 58062 Config: library 58063 .IP \[bu] 2 58064 Env Var: RCLONE_SEAFILE_LIBRARY 58065 .IP \[bu] 2 58066 Type: string 58067 .IP \[bu] 2 58068 Required: false 58069 .SS --seafile-library-key 58070 .PP 58071 Library password (for encrypted libraries only). 58072 .PP 58073 Leave blank if you pass it through the command line. 58074 .PP 58075 \f[B]NB\f[R] Input to this must be obscured - see rclone 58076 obscure (https://rclone.org/commands/rclone_obscure/). 58077 .PP 58078 Properties: 58079 .IP \[bu] 2 58080 Config: library_key 58081 .IP \[bu] 2 58082 Env Var: RCLONE_SEAFILE_LIBRARY_KEY 58083 .IP \[bu] 2 58084 Type: string 58085 .IP \[bu] 2 58086 Required: false 58087 .SS --seafile-auth-token 58088 .PP 58089 Authentication token. 58090 .PP 58091 Properties: 58092 .IP \[bu] 2 58093 Config: auth_token 58094 .IP \[bu] 2 58095 Env Var: RCLONE_SEAFILE_AUTH_TOKEN 58096 .IP \[bu] 2 58097 Type: string 58098 .IP \[bu] 2 58099 Required: false 58100 .SS Advanced options 58101 .PP 58102 Here are the Advanced options specific to seafile (seafile). 58103 .SS --seafile-create-library 58104 .PP 58105 Should rclone create a library if it doesn\[aq]t exist. 58106 .PP 58107 Properties: 58108 .IP \[bu] 2 58109 Config: create_library 58110 .IP \[bu] 2 58111 Env Var: RCLONE_SEAFILE_CREATE_LIBRARY 58112 .IP \[bu] 2 58113 Type: bool 58114 .IP \[bu] 2 58115 Default: false 58116 .SS --seafile-encoding 58117 .PP 58118 The encoding for the backend. 58119 .PP 58120 See the encoding section in the 58121 overview (https://rclone.org/overview/#encoding) for more info. 58122 .PP 58123 Properties: 58124 .IP \[bu] 2 58125 Config: encoding 58126 .IP \[bu] 2 58127 Env Var: RCLONE_SEAFILE_ENCODING 58128 .IP \[bu] 2 58129 Type: Encoding 58130 .IP \[bu] 2 58131 Default: Slash,DoubleQuote,BackSlash,Ctl,InvalidUtf8 58132 .SS --seafile-description 58133 .PP 58134 Description of the remote 58135 .PP 58136 Properties: 58137 .IP \[bu] 2 58138 Config: description 58139 .IP \[bu] 2 58140 Env Var: RCLONE_SEAFILE_DESCRIPTION 58141 .IP \[bu] 2 58142 Type: string 58143 .IP \[bu] 2 58144 Required: false 58145 .SH SFTP 58146 .PP 58147 SFTP is the Secure (or SSH) File Transfer 58148 Protocol (https://en.wikipedia.org/wiki/SSH_File_Transfer_Protocol). 58149 .PP 58150 The SFTP backend can be used with a number of different providers: 58151 .IP \[bu] 2 58152 Hetzner Storage Box 58153 .IP \[bu] 2 58154 rsync.net 58155 .PP 58156 SFTP runs over SSH v2 and is installed as standard with most modern SSH 58157 installations. 58158 .PP 58159 Paths are specified as \f[C]remote:path\f[R]. 58160 If the path does not begin with a \f[C]/\f[R] it is relative to the home 58161 directory of the user. 58162 An empty path \f[C]remote:\f[R] refers to the user\[aq]s home directory. 58163 For example, \f[C]rclone lsd remote:\f[R] would list the home directory 58164 of the user configured in the rclone remote config 58165 (\f[C]i.e /home/sftpuser\f[R]). 58166 However, \f[C]rclone lsd remote:/\f[R] would list the root directory for 58167 remote machine (i.e. 58168 \f[C]/\f[R]) 58169 .PP 58170 Note that some SFTP servers will need the leading / - Synology is a good 58171 example of this. 58172 rsync.net and Hetzner, on the other hand, requires users to OMIT the 58173 leading /. 58174 .PP 58175 Note that by default rclone will try to execute shell commands on the 58176 server, see shell access considerations. 58177 .SS Configuration 58178 .PP 58179 Here is an example of making an SFTP configuration. 58180 First run 58181 .IP 58182 .nf 58183 \f[C] 58184 rclone config 58185 \f[R] 58186 .fi 58187 .PP 58188 This will guide you through an interactive setup process. 58189 .IP 58190 .nf 58191 \f[C] 58192 No remotes found, make a new one? 58193 n) New remote 58194 s) Set configuration password 58195 q) Quit config 58196 n/s/q> n 58197 name> remote 58198 Type of storage to configure. 58199 Choose a number from below, or type in your own value 58200 [snip] 58201 XX / SSH/SFTP 58202 \[rs] \[dq]sftp\[dq] 58203 [snip] 58204 Storage> sftp 58205 SSH host to connect to 58206 Choose a number from below, or type in your own value 58207 1 / Connect to example.com 58208 \[rs] \[dq]example.com\[dq] 58209 host> example.com 58210 SSH username 58211 Enter a string value. Press Enter for the default (\[dq]$USER\[dq]). 58212 user> sftpuser 58213 SSH port number 58214 Enter a signed integer. Press Enter for the default (22). 58215 port> 58216 SSH password, leave blank to use ssh-agent. 58217 y) Yes type in my own password 58218 g) Generate random password 58219 n) No leave this optional password blank 58220 y/g/n> n 58221 Path to unencrypted PEM-encoded private key file, leave blank to use ssh-agent. 58222 key_file> 58223 Remote config 58224 -------------------- 58225 [remote] 58226 host = example.com 58227 user = sftpuser 58228 port = 58229 pass = 58230 key_file = 58231 -------------------- 58232 y) Yes this is OK 58233 e) Edit this remote 58234 d) Delete this remote 58235 y/e/d> y 58236 \f[R] 58237 .fi 58238 .PP 58239 This remote is called \f[C]remote\f[R] and can now be used like this: 58240 .PP 58241 See all directories in the home directory 58242 .IP 58243 .nf 58244 \f[C] 58245 rclone lsd remote: 58246 \f[R] 58247 .fi 58248 .PP 58249 See all directories in the root directory 58250 .IP 58251 .nf 58252 \f[C] 58253 rclone lsd remote:/ 58254 \f[R] 58255 .fi 58256 .PP 58257 Make a new directory 58258 .IP 58259 .nf 58260 \f[C] 58261 rclone mkdir remote:path/to/directory 58262 \f[R] 58263 .fi 58264 .PP 58265 List the contents of a directory 58266 .IP 58267 .nf 58268 \f[C] 58269 rclone ls remote:path/to/directory 58270 \f[R] 58271 .fi 58272 .PP 58273 Sync \f[C]/home/local/directory\f[R] to the remote directory, deleting 58274 any excess files in the directory. 58275 .IP 58276 .nf 58277 \f[C] 58278 rclone sync --interactive /home/local/directory remote:directory 58279 \f[R] 58280 .fi 58281 .PP 58282 Mount the remote path \f[C]/srv/www-data/\f[R] to the local path 58283 \f[C]/mnt/www-data\f[R] 58284 .IP 58285 .nf 58286 \f[C] 58287 rclone mount remote:/srv/www-data/ /mnt/www-data 58288 \f[R] 58289 .fi 58290 .SS SSH Authentication 58291 .PP 58292 The SFTP remote supports three authentication methods: 58293 .IP \[bu] 2 58294 Password 58295 .IP \[bu] 2 58296 Key file, including certificate signed keys 58297 .IP \[bu] 2 58298 ssh-agent 58299 .PP 58300 Key files should be PEM-encoded private key files. 58301 For instance \f[C]/home/$USER/.ssh/id_rsa\f[R]. 58302 Only unencrypted OpenSSH or PEM encrypted files are supported. 58303 .PP 58304 The key file can be specified in either an external file (key_file) or 58305 contained within the rclone config file (key_pem). 58306 If using key_pem in the config file, the entry should be on a single 58307 line with new line (\[aq]\[aq] or \[aq]\[aq]) separating lines. 58308 i.e. 58309 .IP 58310 .nf 58311 \f[C] 58312 key_pem = -----BEGIN RSA PRIVATE KEY-----\[rs]nMaMbaIXtE\[rs]n0gAMbMbaSsd\[rs]nMbaass\[rs]n-----END RSA PRIVATE KEY----- 58313 \f[R] 58314 .fi 58315 .PP 58316 This will generate it correctly for key_pem for use in the config: 58317 .IP 58318 .nf 58319 \f[C] 58320 awk \[aq]{printf \[dq]%s\[rs]\[rs]n\[dq], $0}\[aq] < \[ti]/.ssh/id_rsa 58321 \f[R] 58322 .fi 58323 .PP 58324 If you don\[aq]t specify \f[C]pass\f[R], \f[C]key_file\f[R], or 58325 \f[C]key_pem\f[R] or \f[C]ask_password\f[R] then rclone will attempt to 58326 contact an ssh-agent. 58327 You can also specify \f[C]key_use_agent\f[R] to force the usage of an 58328 ssh-agent. 58329 In this case \f[C]key_file\f[R] or \f[C]key_pem\f[R] can also be 58330 specified to force the usage of a specific key in the ssh-agent. 58331 .PP 58332 Using an ssh-agent is the only way to load encrypted OpenSSH keys at the 58333 moment. 58334 .PP 58335 If you set the \f[C]ask_password\f[R] option, rclone will prompt for a 58336 password when needed and no password has been configured. 58337 .SS Certificate-signed keys 58338 .PP 58339 With traditional key-based authentication, you configure your private 58340 key only, and the public key built into it will be used during the 58341 authentication process. 58342 .PP 58343 If you have a certificate you may use it to sign your public key, 58344 creating a separate SSH user certificate that should be used instead of 58345 the plain public key extracted from the private key. 58346 Then you must provide the path to the user certificate public key file 58347 in \f[C]pubkey_file\f[R]. 58348 .PP 58349 Note: This is not the traditional public key paired with your private 58350 key, typically saved as \f[C]/home/$USER/.ssh/id_rsa.pub\f[R]. 58351 Setting this path in \f[C]pubkey_file\f[R] will not work. 58352 .PP 58353 Example: 58354 .IP 58355 .nf 58356 \f[C] 58357 [remote] 58358 type = sftp 58359 host = example.com 58360 user = sftpuser 58361 key_file = \[ti]/id_rsa 58362 pubkey_file = \[ti]/id_rsa-cert.pub 58363 \f[R] 58364 .fi 58365 .PP 58366 If you concatenate a cert with a private key then you can specify the 58367 merged file in both places. 58368 .PP 58369 Note: the cert must come first in the file. 58370 e.g. 58371 .IP 58372 .nf 58373 \f[C] 58374 cat id_rsa-cert.pub id_rsa > merged_key 58375 \f[R] 58376 .fi 58377 .SS Host key validation 58378 .PP 58379 By default rclone will not check the server\[aq]s host key for 58380 validation. 58381 This can allow an attacker to replace a server with their own and if you 58382 use password authentication then this can lead to that password being 58383 exposed. 58384 .PP 58385 Host key matching, using standard \f[C]known_hosts\f[R] files can be 58386 turned on by enabling the \f[C]known_hosts_file\f[R] option. 58387 This can point to the file maintained by \f[C]OpenSSH\f[R] or can point 58388 to a unique file. 58389 .PP 58390 e.g. 58391 using the OpenSSH \f[C]known_hosts\f[R] file: 58392 .IP 58393 .nf 58394 \f[C] 58395 [remote] 58396 type = sftp 58397 host = example.com 58398 user = sftpuser 58399 pass = 58400 known_hosts_file = \[ti]/.ssh/known_hosts 58401 \f[R] 58402 .fi 58403 .PP 58404 Alternatively you can create your own known hosts file like this: 58405 .IP 58406 .nf 58407 \f[C] 58408 ssh-keyscan -t dsa,rsa,ecdsa,ed25519 example.com >> known_hosts 58409 \f[R] 58410 .fi 58411 .PP 58412 There are some limitations: 58413 .IP \[bu] 2 58414 \f[C]rclone\f[R] will not \f[I]manage\f[R] this file for you. 58415 If the key is missing or wrong then the connection will be refused. 58416 .IP \[bu] 2 58417 If the server is set up for a certificate host key then the entry in the 58418 \f[C]known_hosts\f[R] file \f[I]must\f[R] be the 58419 \f[C]\[at]cert-authority\f[R] entry for the CA 58420 .PP 58421 If the host key provided by the server does not match the one in the 58422 file (or is missing) then the connection will be aborted and an error 58423 returned such as 58424 .IP 58425 .nf 58426 \f[C] 58427 NewFs: couldn\[aq]t connect SSH: ssh: handshake failed: knownhosts: key mismatch 58428 \f[R] 58429 .fi 58430 .PP 58431 or 58432 .IP 58433 .nf 58434 \f[C] 58435 NewFs: couldn\[aq]t connect SSH: ssh: handshake failed: knownhosts: key is unknown 58436 \f[R] 58437 .fi 58438 .PP 58439 If you see an error such as 58440 .IP 58441 .nf 58442 \f[C] 58443 NewFs: couldn\[aq]t connect SSH: ssh: handshake failed: ssh: no authorities for hostname: example.com:22 58444 \f[R] 58445 .fi 58446 .PP 58447 then it is likely the server has presented a CA signed host certificate 58448 and you will need to add the appropriate \f[C]\[at]cert-authority\f[R] 58449 entry. 58450 .PP 58451 The \f[C]known_hosts_file\f[R] setting can be set during 58452 \f[C]rclone config\f[R] as an advanced option. 58453 .SS ssh-agent on macOS 58454 .PP 58455 Note that there seem to be various problems with using an ssh-agent on 58456 macOS due to recent changes in the OS. 58457 The most effective work-around seems to be to start an ssh-agent in each 58458 session, e.g. 58459 .IP 58460 .nf 58461 \f[C] 58462 eval \[ga]ssh-agent -s\[ga] && ssh-add -A 58463 \f[R] 58464 .fi 58465 .PP 58466 And then at the end of the session 58467 .IP 58468 .nf 58469 \f[C] 58470 eval \[ga]ssh-agent -k\[ga] 58471 \f[R] 58472 .fi 58473 .PP 58474 These commands can be used in scripts of course. 58475 .SS Shell access 58476 .PP 58477 Some functionality of the SFTP backend relies on remote shell access, 58478 and the possibility to execute commands. 58479 This includes checksum, and in some cases also about. 58480 The shell commands that must be executed may be different on different 58481 type of shells, and also quoting/escaping of file path arguments 58482 containing special characters may be different. 58483 Rclone therefore needs to know what type of shell it is, and if shell 58484 access is available at all. 58485 .PP 58486 Most servers run on some version of Unix, and then a basic Unix shell 58487 can be assumed, without further distinction. 58488 Windows 10, Server 2019, and later can also run a SSH server, which is a 58489 port of OpenSSH (see official installation 58490 guide (https://docs.microsoft.com/en-us/windows-server/administration/openssh/openssh_install_firstuse)). 58491 On a Windows server the shell handling is different: Although it can 58492 also be set up to use a Unix type shell, e.g. 58493 Cygwin bash, the default is to use Windows Command Prompt (cmd.exe), and 58494 PowerShell is a recommended alternative. 58495 All of these have behave differently, which rclone must handle. 58496 .PP 58497 Rclone tries to auto-detect what type of shell is used on the server, 58498 first time you access the SFTP remote. 58499 If a remote shell session is successfully created, it will look for 58500 indications that it is CMD or PowerShell, with fall-back to Unix if not 58501 something else is detected. 58502 If unable to even create a remote shell session, then shell command 58503 execution will be disabled entirely. 58504 The result is stored in the SFTP remote configuration, in option 58505 \f[C]shell_type\f[R], so that the auto-detection only have to be 58506 performed once. 58507 If you manually set a value for this option before first run, the 58508 auto-detection will be skipped, and if you set a different value later 58509 this will override any existing. 58510 Value \f[C]none\f[R] can be set to avoid any attempts at executing shell 58511 commands, e.g. 58512 if this is not allowed on the server. 58513 .PP 58514 When the server is rclone serve 58515 sftp (https://rclone.org/commands/rclone_serve_sftp/), the rclone SFTP 58516 remote will detect this as a Unix type shell - even if it is running on 58517 Windows. 58518 This server does not actually have a shell, but it accepts input 58519 commands matching the specific ones that the SFTP backend relies on for 58520 Unix shells, e.g. 58521 \f[C]md5sum\f[R] and \f[C]df\f[R]. 58522 Also it handles the string escape rules used for Unix shell. 58523 Treating it as a Unix type shell from a SFTP remote will therefore 58524 always be correct, and support all features. 58525 .SS Shell access considerations 58526 .PP 58527 The shell type auto-detection logic, described above, means that by 58528 default rclone will try to run a shell command the first time a new sftp 58529 remote is accessed. 58530 If you configure a sftp remote without a config file, e.g. 58531 an on the fly (https://rclone.org/docs/#backend-path-to-dir%5D) remote, 58532 rclone will have nowhere to store the result, and it will re-run the 58533 command on every access. 58534 To avoid this you should explicitly set the \f[C]shell_type\f[R] option 58535 to the correct value, or to \f[C]none\f[R] if you want to prevent rclone 58536 from executing any remote shell commands. 58537 .PP 58538 It is also important to note that, since the shell type decides how 58539 quoting and escaping of file paths used as command-line arguments are 58540 performed, configuring the wrong shell type may leave you exposed to 58541 command injection exploits. 58542 Make sure to confirm the auto-detected shell type, or explicitly set the 58543 shell type you know is correct, or disable shell access until you know. 58544 .SS Checksum 58545 .PP 58546 SFTP does not natively support checksums (file hash), but rclone is able 58547 to use checksumming if the same login has shell access, and can execute 58548 remote commands. 58549 If there is a command that can calculate compatible checksums on the 58550 remote system, Rclone can then be configured to execute this whenever a 58551 checksum is needed, and read back the results. 58552 Currently MD5 and SHA-1 are supported. 58553 .PP 58554 Normally this requires an external utility being available on the 58555 server. 58556 By default rclone will try commands \f[C]md5sum\f[R], \f[C]md5\f[R] and 58557 \f[C]rclone md5sum\f[R] for MD5 checksums, and the first one found 58558 usable will be picked. 58559 Same with \f[C]sha1sum\f[R], \f[C]sha1\f[R] and \f[C]rclone sha1sum\f[R] 58560 commands for SHA-1 checksums. 58561 These utilities normally need to be in the remote\[aq]s PATH to be 58562 found. 58563 .PP 58564 In some cases the shell itself is capable of calculating checksums. 58565 PowerShell is an example of such a shell. 58566 If rclone detects that the remote shell is PowerShell, which means it 58567 most probably is a Windows OpenSSH server, rclone will use a predefined 58568 script block to produce the checksums when no external checksum commands 58569 are found (see shell access). 58570 This assumes PowerShell version 4.0 or newer. 58571 .PP 58572 The options \f[C]md5sum_command\f[R] and \f[C]sha1_command\f[R] can be 58573 used to customize the command to be executed for calculation of 58574 checksums. 58575 You can for example set a specific path to where md5sum and sha1sum 58576 executables are located, or use them to specify some other tools that 58577 print checksums in compatible format. 58578 The value can include command-line arguments, or even shell script 58579 blocks as with PowerShell. 58580 Rclone has subcommands 58581 md5sum (https://rclone.org/commands/rclone_md5sum/) and 58582 sha1sum (https://rclone.org/commands/rclone_sha1sum/) that use 58583 compatible format, which means if you have an rclone executable on the 58584 server it can be used. 58585 As mentioned above, they will be automatically picked up if found in 58586 PATH, but if not you can set something like 58587 \f[C]/path/to/rclone md5sum\f[R] as the value of option 58588 \f[C]md5sum_command\f[R] to make sure a specific executable is used. 58589 .PP 58590 Remote checksumming is recommended and enabled by default. 58591 First time rclone is using a SFTP remote, if options 58592 \f[C]md5sum_command\f[R] or \f[C]sha1_command\f[R] are not set, it will 58593 check if any of the default commands for each of them, as described 58594 above, can be used. 58595 The result will be saved in the remote configuration, so next time it 58596 will use the same. 58597 Value \f[C]none\f[R] will be set if none of the default commands could 58598 be used for a specific algorithm, and this algorithm will not be 58599 supported by the remote. 58600 .PP 58601 Disabling the checksumming may be required if you are connecting to SFTP 58602 servers which are not under your control, and to which the execution of 58603 remote shell commands is prohibited. 58604 Set the configuration option \f[C]disable_hashcheck\f[R] to 58605 \f[C]true\f[R] to disable checksumming entirely, or set 58606 \f[C]shell_type\f[R] to \f[C]none\f[R] to disable all functionality 58607 based on remote shell command execution. 58608 .SS Modification times and hashes 58609 .PP 58610 Modified times are stored on the server to 1 second precision. 58611 .PP 58612 Modified times are used in syncing and are fully supported. 58613 .PP 58614 Some SFTP servers disable setting/modifying the file modification time 58615 after upload (for example, certain configurations of ProFTPd with 58616 mod_sftp). 58617 If you are using one of these servers, you can set the option 58618 \f[C]set_modtime = false\f[R] in your RClone backend configuration to 58619 disable this behaviour. 58620 .SS About command 58621 .PP 58622 The \f[C]about\f[R] command returns the total space, free space, and 58623 used space on the remote for the disk of the specified path on the 58624 remote or, if not set, the disk of the root on the remote. 58625 .PP 58626 SFTP usually supports the 58627 about (https://rclone.org/commands/rclone_about/) command, but it 58628 depends on the server. 58629 If the server implements the vendor-specific VFS statistics extension, 58630 which is normally the case with OpenSSH instances, it will be used. 58631 If not, but the same login has access to a Unix shell, where the 58632 \f[C]df\f[R] command is available (e.g. 58633 in the remote\[aq]s PATH), then this will be used instead. 58634 If the server shell is PowerShell, probably with a Windows OpenSSH 58635 server, rclone will use a built-in shell command (see shell access). 58636 If none of the above is applicable, \f[C]about\f[R] will fail. 58637 .SS Standard options 58638 .PP 58639 Here are the Standard options specific to sftp (SSH/SFTP). 58640 .SS --sftp-host 58641 .PP 58642 SSH host to connect to. 58643 .PP 58644 E.g. 58645 \[dq]example.com\[dq]. 58646 .PP 58647 Properties: 58648 .IP \[bu] 2 58649 Config: host 58650 .IP \[bu] 2 58651 Env Var: RCLONE_SFTP_HOST 58652 .IP \[bu] 2 58653 Type: string 58654 .IP \[bu] 2 58655 Required: true 58656 .SS --sftp-user 58657 .PP 58658 SSH username. 58659 .PP 58660 Properties: 58661 .IP \[bu] 2 58662 Config: user 58663 .IP \[bu] 2 58664 Env Var: RCLONE_SFTP_USER 58665 .IP \[bu] 2 58666 Type: string 58667 .IP \[bu] 2 58668 Default: \[dq]$USER\[dq] 58669 .SS --sftp-port 58670 .PP 58671 SSH port number. 58672 .PP 58673 Properties: 58674 .IP \[bu] 2 58675 Config: port 58676 .IP \[bu] 2 58677 Env Var: RCLONE_SFTP_PORT 58678 .IP \[bu] 2 58679 Type: int 58680 .IP \[bu] 2 58681 Default: 22 58682 .SS --sftp-pass 58683 .PP 58684 SSH password, leave blank to use ssh-agent. 58685 .PP 58686 \f[B]NB\f[R] Input to this must be obscured - see rclone 58687 obscure (https://rclone.org/commands/rclone_obscure/). 58688 .PP 58689 Properties: 58690 .IP \[bu] 2 58691 Config: pass 58692 .IP \[bu] 2 58693 Env Var: RCLONE_SFTP_PASS 58694 .IP \[bu] 2 58695 Type: string 58696 .IP \[bu] 2 58697 Required: false 58698 .SS --sftp-key-pem 58699 .PP 58700 Raw PEM-encoded private key. 58701 .PP 58702 If specified, will override key_file parameter. 58703 .PP 58704 Properties: 58705 .IP \[bu] 2 58706 Config: key_pem 58707 .IP \[bu] 2 58708 Env Var: RCLONE_SFTP_KEY_PEM 58709 .IP \[bu] 2 58710 Type: string 58711 .IP \[bu] 2 58712 Required: false 58713 .SS --sftp-key-file 58714 .PP 58715 Path to PEM-encoded private key file. 58716 .PP 58717 Leave blank or set key-use-agent to use ssh-agent. 58718 .PP 58719 Leading \f[C]\[ti]\f[R] will be expanded in the file name as will 58720 environment variables such as \f[C]${RCLONE_CONFIG_DIR}\f[R]. 58721 .PP 58722 Properties: 58723 .IP \[bu] 2 58724 Config: key_file 58725 .IP \[bu] 2 58726 Env Var: RCLONE_SFTP_KEY_FILE 58727 .IP \[bu] 2 58728 Type: string 58729 .IP \[bu] 2 58730 Required: false 58731 .SS --sftp-key-file-pass 58732 .PP 58733 The passphrase to decrypt the PEM-encoded private key file. 58734 .PP 58735 Only PEM encrypted key files (old OpenSSH format) are supported. 58736 Encrypted keys in the new OpenSSH format can\[aq]t be used. 58737 .PP 58738 \f[B]NB\f[R] Input to this must be obscured - see rclone 58739 obscure (https://rclone.org/commands/rclone_obscure/). 58740 .PP 58741 Properties: 58742 .IP \[bu] 2 58743 Config: key_file_pass 58744 .IP \[bu] 2 58745 Env Var: RCLONE_SFTP_KEY_FILE_PASS 58746 .IP \[bu] 2 58747 Type: string 58748 .IP \[bu] 2 58749 Required: false 58750 .SS --sftp-pubkey-file 58751 .PP 58752 Optional path to public key file. 58753 .PP 58754 Set this if you have a signed certificate you want to use for 58755 authentication. 58756 .PP 58757 Leading \f[C]\[ti]\f[R] will be expanded in the file name as will 58758 environment variables such as \f[C]${RCLONE_CONFIG_DIR}\f[R]. 58759 .PP 58760 Properties: 58761 .IP \[bu] 2 58762 Config: pubkey_file 58763 .IP \[bu] 2 58764 Env Var: RCLONE_SFTP_PUBKEY_FILE 58765 .IP \[bu] 2 58766 Type: string 58767 .IP \[bu] 2 58768 Required: false 58769 .SS --sftp-key-use-agent 58770 .PP 58771 When set forces the usage of the ssh-agent. 58772 .PP 58773 When key-file is also set, the \[dq].pub\[dq] file of the specified 58774 key-file is read and only the associated key is requested from the 58775 ssh-agent. 58776 This allows to avoid 58777 \f[C]Too many authentication failures for *username*\f[R] errors when 58778 the ssh-agent contains many keys. 58779 .PP 58780 Properties: 58781 .IP \[bu] 2 58782 Config: key_use_agent 58783 .IP \[bu] 2 58784 Env Var: RCLONE_SFTP_KEY_USE_AGENT 58785 .IP \[bu] 2 58786 Type: bool 58787 .IP \[bu] 2 58788 Default: false 58789 .SS --sftp-use-insecure-cipher 58790 .PP 58791 Enable the use of insecure ciphers and key exchange methods. 58792 .PP 58793 This enables the use of the following insecure ciphers and key exchange 58794 methods: 58795 .IP \[bu] 2 58796 aes128-cbc 58797 .IP \[bu] 2 58798 aes192-cbc 58799 .IP \[bu] 2 58800 aes256-cbc 58801 .IP \[bu] 2 58802 3des-cbc 58803 .IP \[bu] 2 58804 diffie-hellman-group-exchange-sha256 58805 .IP \[bu] 2 58806 diffie-hellman-group-exchange-sha1 58807 .PP 58808 Those algorithms are insecure and may allow plaintext data to be 58809 recovered by an attacker. 58810 .PP 58811 This must be false if you use either ciphers or key_exchange advanced 58812 options. 58813 .PP 58814 Properties: 58815 .IP \[bu] 2 58816 Config: use_insecure_cipher 58817 .IP \[bu] 2 58818 Env Var: RCLONE_SFTP_USE_INSECURE_CIPHER 58819 .IP \[bu] 2 58820 Type: bool 58821 .IP \[bu] 2 58822 Default: false 58823 .IP \[bu] 2 58824 Examples: 58825 .RS 2 58826 .IP \[bu] 2 58827 \[dq]false\[dq] 58828 .RS 2 58829 .IP \[bu] 2 58830 Use default Cipher list. 58831 .RE 58832 .IP \[bu] 2 58833 \[dq]true\[dq] 58834 .RS 2 58835 .IP \[bu] 2 58836 Enables the use of the aes128-cbc cipher and 58837 diffie-hellman-group-exchange-sha256, diffie-hellman-group-exchange-sha1 58838 key exchange. 58839 .RE 58840 .RE 58841 .SS --sftp-disable-hashcheck 58842 .PP 58843 Disable the execution of SSH commands to determine if remote file 58844 hashing is available. 58845 .PP 58846 Leave blank or set to false to enable hashing (recommended), set to true 58847 to disable hashing. 58848 .PP 58849 Properties: 58850 .IP \[bu] 2 58851 Config: disable_hashcheck 58852 .IP \[bu] 2 58853 Env Var: RCLONE_SFTP_DISABLE_HASHCHECK 58854 .IP \[bu] 2 58855 Type: bool 58856 .IP \[bu] 2 58857 Default: false 58858 .SS --sftp-ssh 58859 .PP 58860 Path and arguments to external ssh binary. 58861 .PP 58862 Normally rclone will use its internal ssh library to connect to the SFTP 58863 server. 58864 However it does not implement all possible ssh options so it may be 58865 desirable to use an external ssh binary. 58866 .PP 58867 Rclone ignores all the internal config if you use this option and 58868 expects you to configure the ssh binary with the user/host/port and any 58869 other options you need. 58870 .PP 58871 \f[B]Important\f[R] The ssh command must log in without asking for a 58872 password so needs to be configured with keys or certificates. 58873 .PP 58874 Rclone will run the command supplied either with the additional 58875 arguments \[dq]-s sftp\[dq] to access the SFTP subsystem or with 58876 commands such as \[dq]md5sum /path/to/file\[dq] appended to read 58877 checksums. 58878 .PP 58879 Any arguments with spaces in should be surrounded by \[dq]double 58880 quotes\[dq]. 58881 .PP 58882 An example setting might be: 58883 .IP 58884 .nf 58885 \f[C] 58886 ssh -o ServerAliveInterval=20 user\[at]example.com 58887 \f[R] 58888 .fi 58889 .PP 58890 Note that when using an external ssh binary rclone makes a new ssh 58891 connection for every hash it calculates. 58892 .PP 58893 Properties: 58894 .IP \[bu] 2 58895 Config: ssh 58896 .IP \[bu] 2 58897 Env Var: RCLONE_SFTP_SSH 58898 .IP \[bu] 2 58899 Type: SpaceSepList 58900 .IP \[bu] 2 58901 Default: 58902 .SS Advanced options 58903 .PP 58904 Here are the Advanced options specific to sftp (SSH/SFTP). 58905 .SS --sftp-known-hosts-file 58906 .PP 58907 Optional path to known_hosts file. 58908 .PP 58909 Set this value to enable server host key validation. 58910 .PP 58911 Leading \f[C]\[ti]\f[R] will be expanded in the file name as will 58912 environment variables such as \f[C]${RCLONE_CONFIG_DIR}\f[R]. 58913 .PP 58914 Properties: 58915 .IP \[bu] 2 58916 Config: known_hosts_file 58917 .IP \[bu] 2 58918 Env Var: RCLONE_SFTP_KNOWN_HOSTS_FILE 58919 .IP \[bu] 2 58920 Type: string 58921 .IP \[bu] 2 58922 Required: false 58923 .IP \[bu] 2 58924 Examples: 58925 .RS 2 58926 .IP \[bu] 2 58927 \[dq]\[ti]/.ssh/known_hosts\[dq] 58928 .RS 2 58929 .IP \[bu] 2 58930 Use OpenSSH\[aq]s known_hosts file. 58931 .RE 58932 .RE 58933 .SS --sftp-ask-password 58934 .PP 58935 Allow asking for SFTP password when needed. 58936 .PP 58937 If this is set and no password is supplied then rclone will: - ask for a 58938 password - not contact the ssh agent 58939 .PP 58940 Properties: 58941 .IP \[bu] 2 58942 Config: ask_password 58943 .IP \[bu] 2 58944 Env Var: RCLONE_SFTP_ASK_PASSWORD 58945 .IP \[bu] 2 58946 Type: bool 58947 .IP \[bu] 2 58948 Default: false 58949 .SS --sftp-path-override 58950 .PP 58951 Override path used by SSH shell commands. 58952 .PP 58953 This allows checksum calculation when SFTP and SSH paths are different. 58954 This issue affects among others Synology NAS boxes. 58955 .PP 58956 E.g. 58957 if shared folders can be found in directories representing volumes: 58958 .IP 58959 .nf 58960 \f[C] 58961 rclone sync /home/local/directory remote:/directory --sftp-path-override /volume2/directory 58962 \f[R] 58963 .fi 58964 .PP 58965 E.g. 58966 if home directory can be found in a shared folder called \[dq]home\[dq]: 58967 .IP 58968 .nf 58969 \f[C] 58970 rclone sync /home/local/directory remote:/home/directory --sftp-path-override /volume1/homes/USER/directory 58971 \f[R] 58972 .fi 58973 .PP 58974 To specify only the path to the SFTP remote\[aq]s root, and allow rclone 58975 to add any relative subpaths automatically (including 58976 unwrapping/decrypting remotes as necessary), add the \[aq]\[at]\[aq] 58977 character to the beginning of the path. 58978 .PP 58979 E.g. 58980 the first example above could be rewritten as: 58981 .IP 58982 .nf 58983 \f[C] 58984 rclone sync /home/local/directory remote:/directory --sftp-path-override \[at]/volume2 58985 \f[R] 58986 .fi 58987 .PP 58988 Note that when using this method with Synology \[dq]home\[dq] folders, 58989 the full \[dq]/homes/USER\[dq] path should be specified instead of 58990 \[dq]/home\[dq]. 58991 .PP 58992 E.g. 58993 the second example above should be rewritten as: 58994 .IP 58995 .nf 58996 \f[C] 58997 rclone sync /home/local/directory remote:/homes/USER/directory --sftp-path-override \[at]/volume1 58998 \f[R] 58999 .fi 59000 .PP 59001 Properties: 59002 .IP \[bu] 2 59003 Config: path_override 59004 .IP \[bu] 2 59005 Env Var: RCLONE_SFTP_PATH_OVERRIDE 59006 .IP \[bu] 2 59007 Type: string 59008 .IP \[bu] 2 59009 Required: false 59010 .SS --sftp-set-modtime 59011 .PP 59012 Set the modified time on the remote if set. 59013 .PP 59014 Properties: 59015 .IP \[bu] 2 59016 Config: set_modtime 59017 .IP \[bu] 2 59018 Env Var: RCLONE_SFTP_SET_MODTIME 59019 .IP \[bu] 2 59020 Type: bool 59021 .IP \[bu] 2 59022 Default: true 59023 .SS --sftp-shell-type 59024 .PP 59025 The type of SSH shell on remote server, if any. 59026 .PP 59027 Leave blank for autodetect. 59028 .PP 59029 Properties: 59030 .IP \[bu] 2 59031 Config: shell_type 59032 .IP \[bu] 2 59033 Env Var: RCLONE_SFTP_SHELL_TYPE 59034 .IP \[bu] 2 59035 Type: string 59036 .IP \[bu] 2 59037 Required: false 59038 .IP \[bu] 2 59039 Examples: 59040 .RS 2 59041 .IP \[bu] 2 59042 \[dq]none\[dq] 59043 .RS 2 59044 .IP \[bu] 2 59045 No shell access 59046 .RE 59047 .IP \[bu] 2 59048 \[dq]unix\[dq] 59049 .RS 2 59050 .IP \[bu] 2 59051 Unix shell 59052 .RE 59053 .IP \[bu] 2 59054 \[dq]powershell\[dq] 59055 .RS 2 59056 .IP \[bu] 2 59057 PowerShell 59058 .RE 59059 .IP \[bu] 2 59060 \[dq]cmd\[dq] 59061 .RS 2 59062 .IP \[bu] 2 59063 Windows Command Prompt 59064 .RE 59065 .RE 59066 .SS --sftp-md5sum-command 59067 .PP 59068 The command used to read md5 hashes. 59069 .PP 59070 Leave blank for autodetect. 59071 .PP 59072 Properties: 59073 .IP \[bu] 2 59074 Config: md5sum_command 59075 .IP \[bu] 2 59076 Env Var: RCLONE_SFTP_MD5SUM_COMMAND 59077 .IP \[bu] 2 59078 Type: string 59079 .IP \[bu] 2 59080 Required: false 59081 .SS --sftp-sha1sum-command 59082 .PP 59083 The command used to read sha1 hashes. 59084 .PP 59085 Leave blank for autodetect. 59086 .PP 59087 Properties: 59088 .IP \[bu] 2 59089 Config: sha1sum_command 59090 .IP \[bu] 2 59091 Env Var: RCLONE_SFTP_SHA1SUM_COMMAND 59092 .IP \[bu] 2 59093 Type: string 59094 .IP \[bu] 2 59095 Required: false 59096 .SS --sftp-skip-links 59097 .PP 59098 Set to skip any symlinks and any other non regular files. 59099 .PP 59100 Properties: 59101 .IP \[bu] 2 59102 Config: skip_links 59103 .IP \[bu] 2 59104 Env Var: RCLONE_SFTP_SKIP_LINKS 59105 .IP \[bu] 2 59106 Type: bool 59107 .IP \[bu] 2 59108 Default: false 59109 .SS --sftp-subsystem 59110 .PP 59111 Specifies the SSH2 subsystem on the remote host. 59112 .PP 59113 Properties: 59114 .IP \[bu] 2 59115 Config: subsystem 59116 .IP \[bu] 2 59117 Env Var: RCLONE_SFTP_SUBSYSTEM 59118 .IP \[bu] 2 59119 Type: string 59120 .IP \[bu] 2 59121 Default: \[dq]sftp\[dq] 59122 .SS --sftp-server-command 59123 .PP 59124 Specifies the path or command to run a sftp server on the remote host. 59125 .PP 59126 The subsystem option is ignored when server_command is defined. 59127 .PP 59128 If adding server_command to the configuration file please note that it 59129 should not be enclosed in quotes, since that will make rclone fail. 59130 .PP 59131 A working example is: 59132 .IP 59133 .nf 59134 \f[C] 59135 [remote_name] 59136 type = sftp 59137 server_command = sudo /usr/libexec/openssh/sftp-server 59138 \f[R] 59139 .fi 59140 .PP 59141 Properties: 59142 .IP \[bu] 2 59143 Config: server_command 59144 .IP \[bu] 2 59145 Env Var: RCLONE_SFTP_SERVER_COMMAND 59146 .IP \[bu] 2 59147 Type: string 59148 .IP \[bu] 2 59149 Required: false 59150 .SS --sftp-use-fstat 59151 .PP 59152 If set use fstat instead of stat. 59153 .PP 59154 Some servers limit the amount of open files and calling Stat after 59155 opening the file will throw an error from the server. 59156 Setting this flag will call Fstat instead of Stat which is called on an 59157 already open file handle. 59158 .PP 59159 It has been found that this helps with IBM Sterling SFTP servers which 59160 have \[dq]extractability\[dq] level set to 1 which means only 1 file can 59161 be opened at any given time. 59162 .PP 59163 Properties: 59164 .IP \[bu] 2 59165 Config: use_fstat 59166 .IP \[bu] 2 59167 Env Var: RCLONE_SFTP_USE_FSTAT 59168 .IP \[bu] 2 59169 Type: bool 59170 .IP \[bu] 2 59171 Default: false 59172 .SS --sftp-disable-concurrent-reads 59173 .PP 59174 If set don\[aq]t use concurrent reads. 59175 .PP 59176 Normally concurrent reads are safe to use and not using them will 59177 degrade performance, so this option is disabled by default. 59178 .PP 59179 Some servers limit the amount number of times a file can be downloaded. 59180 Using concurrent reads can trigger this limit, so if you have a server 59181 which returns 59182 .IP 59183 .nf 59184 \f[C] 59185 Failed to copy: file does not exist 59186 \f[R] 59187 .fi 59188 .PP 59189 Then you may need to enable this flag. 59190 .PP 59191 If concurrent reads are disabled, the use_fstat option is ignored. 59192 .PP 59193 Properties: 59194 .IP \[bu] 2 59195 Config: disable_concurrent_reads 59196 .IP \[bu] 2 59197 Env Var: RCLONE_SFTP_DISABLE_CONCURRENT_READS 59198 .IP \[bu] 2 59199 Type: bool 59200 .IP \[bu] 2 59201 Default: false 59202 .SS --sftp-disable-concurrent-writes 59203 .PP 59204 If set don\[aq]t use concurrent writes. 59205 .PP 59206 Normally rclone uses concurrent writes to upload files. 59207 This improves the performance greatly, especially for distant servers. 59208 .PP 59209 This option disables concurrent writes should that be necessary. 59210 .PP 59211 Properties: 59212 .IP \[bu] 2 59213 Config: disable_concurrent_writes 59214 .IP \[bu] 2 59215 Env Var: RCLONE_SFTP_DISABLE_CONCURRENT_WRITES 59216 .IP \[bu] 2 59217 Type: bool 59218 .IP \[bu] 2 59219 Default: false 59220 .SS --sftp-idle-timeout 59221 .PP 59222 Max time before closing idle connections. 59223 .PP 59224 If no connections have been returned to the connection pool in the time 59225 given, rclone will empty the connection pool. 59226 .PP 59227 Set to 0 to keep connections indefinitely. 59228 .PP 59229 Properties: 59230 .IP \[bu] 2 59231 Config: idle_timeout 59232 .IP \[bu] 2 59233 Env Var: RCLONE_SFTP_IDLE_TIMEOUT 59234 .IP \[bu] 2 59235 Type: Duration 59236 .IP \[bu] 2 59237 Default: 1m0s 59238 .SS --sftp-chunk-size 59239 .PP 59240 Upload and download chunk size. 59241 .PP 59242 This controls the maximum size of payload in SFTP protocol packets. 59243 The RFC limits this to 32768 bytes (32k), which is the default. 59244 However, a lot of servers support larger sizes, typically limited to a 59245 maximum total package size of 256k, and setting it larger will increase 59246 transfer speed dramatically on high latency links. 59247 This includes OpenSSH, and, for example, using the value of 255k works 59248 well, leaving plenty of room for overhead while still being within a 59249 total packet size of 256k. 59250 .PP 59251 Make sure to test thoroughly before using a value higher than 32k, and 59252 only use it if you always connect to the same server or after 59253 sufficiently broad testing. 59254 If you get errors such as \[dq]failed to send packet payload: EOF\[dq], 59255 lots of \[dq]connection lost\[dq], or \[dq]corrupted on transfer\[dq], 59256 when copying a larger file, try lowering the value. 59257 The server run by rclone serve sftp sends packets with standard 32k 59258 maximum payload so you must not set a different chunk_size when 59259 downloading files, but it accepts packets up to the 256k total size, so 59260 for uploads the chunk_size can be set as for the OpenSSH example above. 59261 .PP 59262 Properties: 59263 .IP \[bu] 2 59264 Config: chunk_size 59265 .IP \[bu] 2 59266 Env Var: RCLONE_SFTP_CHUNK_SIZE 59267 .IP \[bu] 2 59268 Type: SizeSuffix 59269 .IP \[bu] 2 59270 Default: 32Ki 59271 .SS --sftp-concurrency 59272 .PP 59273 The maximum number of outstanding requests for one file 59274 .PP 59275 This controls the maximum number of outstanding requests for one file. 59276 Increasing it will increase throughput on high latency links at the cost 59277 of using more memory. 59278 .PP 59279 Properties: 59280 .IP \[bu] 2 59281 Config: concurrency 59282 .IP \[bu] 2 59283 Env Var: RCLONE_SFTP_CONCURRENCY 59284 .IP \[bu] 2 59285 Type: int 59286 .IP \[bu] 2 59287 Default: 64 59288 .SS --sftp-set-env 59289 .PP 59290 Environment variables to pass to sftp and commands 59291 .PP 59292 Set environment variables in the form: 59293 .IP 59294 .nf 59295 \f[C] 59296 VAR=value 59297 \f[R] 59298 .fi 59299 .PP 59300 to be passed to the sftp client and to any commands run (eg md5sum). 59301 .PP 59302 Pass multiple variables space separated, eg 59303 .IP 59304 .nf 59305 \f[C] 59306 VAR1=value VAR2=value 59307 \f[R] 59308 .fi 59309 .PP 59310 and pass variables with spaces in quotes, eg 59311 .IP 59312 .nf 59313 \f[C] 59314 \[dq]VAR3=value with space\[dq] \[dq]VAR4=value with space\[dq] VAR5=nospacehere 59315 \f[R] 59316 .fi 59317 .PP 59318 Properties: 59319 .IP \[bu] 2 59320 Config: set_env 59321 .IP \[bu] 2 59322 Env Var: RCLONE_SFTP_SET_ENV 59323 .IP \[bu] 2 59324 Type: SpaceSepList 59325 .IP \[bu] 2 59326 Default: 59327 .SS --sftp-ciphers 59328 .PP 59329 Space separated list of ciphers to be used for session encryption, 59330 ordered by preference. 59331 .PP 59332 At least one must match with server configuration. 59333 This can be checked for example using ssh -Q cipher. 59334 .PP 59335 This must not be set if use_insecure_cipher is true. 59336 .PP 59337 Example: 59338 .IP 59339 .nf 59340 \f[C] 59341 aes128-ctr aes192-ctr aes256-ctr aes128-gcm\[at]openssh.com aes256-gcm\[at]openssh.com 59342 \f[R] 59343 .fi 59344 .PP 59345 Properties: 59346 .IP \[bu] 2 59347 Config: ciphers 59348 .IP \[bu] 2 59349 Env Var: RCLONE_SFTP_CIPHERS 59350 .IP \[bu] 2 59351 Type: SpaceSepList 59352 .IP \[bu] 2 59353 Default: 59354 .SS --sftp-key-exchange 59355 .PP 59356 Space separated list of key exchange algorithms, ordered by preference. 59357 .PP 59358 At least one must match with server configuration. 59359 This can be checked for example using ssh -Q kex. 59360 .PP 59361 This must not be set if use_insecure_cipher is true. 59362 .PP 59363 Example: 59364 .IP 59365 .nf 59366 \f[C] 59367 sntrup761x25519-sha512\[at]openssh.com curve25519-sha256 curve25519-sha256\[at]libssh.org ecdh-sha2-nistp256 59368 \f[R] 59369 .fi 59370 .PP 59371 Properties: 59372 .IP \[bu] 2 59373 Config: key_exchange 59374 .IP \[bu] 2 59375 Env Var: RCLONE_SFTP_KEY_EXCHANGE 59376 .IP \[bu] 2 59377 Type: SpaceSepList 59378 .IP \[bu] 2 59379 Default: 59380 .SS --sftp-macs 59381 .PP 59382 Space separated list of MACs (message authentication code) algorithms, 59383 ordered by preference. 59384 .PP 59385 At least one must match with server configuration. 59386 This can be checked for example using ssh -Q mac. 59387 .PP 59388 Example: 59389 .IP 59390 .nf 59391 \f[C] 59392 umac-64-etm\[at]openssh.com umac-128-etm\[at]openssh.com hmac-sha2-256-etm\[at]openssh.com 59393 \f[R] 59394 .fi 59395 .PP 59396 Properties: 59397 .IP \[bu] 2 59398 Config: macs 59399 .IP \[bu] 2 59400 Env Var: RCLONE_SFTP_MACS 59401 .IP \[bu] 2 59402 Type: SpaceSepList 59403 .IP \[bu] 2 59404 Default: 59405 .SS --sftp-host-key-algorithms 59406 .PP 59407 Space separated list of host key algorithms, ordered by preference. 59408 .PP 59409 At least one must match with server configuration. 59410 This can be checked for example using ssh -Q HostKeyAlgorithms. 59411 .PP 59412 Note: This can affect the outcome of key negotiation with the server 59413 even if server host key validation is not enabled. 59414 .PP 59415 Example: 59416 .IP 59417 .nf 59418 \f[C] 59419 ssh-ed25519 ssh-rsa ssh-dss 59420 \f[R] 59421 .fi 59422 .PP 59423 Properties: 59424 .IP \[bu] 2 59425 Config: host_key_algorithms 59426 .IP \[bu] 2 59427 Env Var: RCLONE_SFTP_HOST_KEY_ALGORITHMS 59428 .IP \[bu] 2 59429 Type: SpaceSepList 59430 .IP \[bu] 2 59431 Default: 59432 .SS --sftp-socks-proxy 59433 .PP 59434 Socks 5 proxy host. 59435 .PP 59436 Supports the format user:pass\[at]host:port, user\[at]host:port, 59437 host:port. 59438 .PP 59439 Example: 59440 .IP 59441 .nf 59442 \f[C] 59443 myUser:myPass\[at]localhost:9005 59444 \f[R] 59445 .fi 59446 .PP 59447 Properties: 59448 .IP \[bu] 2 59449 Config: socks_proxy 59450 .IP \[bu] 2 59451 Env Var: RCLONE_SFTP_SOCKS_PROXY 59452 .IP \[bu] 2 59453 Type: string 59454 .IP \[bu] 2 59455 Required: false 59456 .SS --sftp-copy-is-hardlink 59457 .PP 59458 Set to enable server side copies using hardlinks. 59459 .PP 59460 The SFTP protocol does not define a copy command so normally server side 59461 copies are not allowed with the sftp backend. 59462 .PP 59463 However the SFTP protocol does support hardlinking, and if you enable 59464 this flag then the sftp backend will support server side copies. 59465 These will be implemented by doing a hardlink from the source to the 59466 destination. 59467 .PP 59468 Not all sftp servers support this. 59469 .PP 59470 Note that hardlinking two files together will use no additional space as 59471 the source and the destination will be the same file. 59472 .PP 59473 This feature may be useful backups made with --copy-dest. 59474 .PP 59475 Properties: 59476 .IP \[bu] 2 59477 Config: copy_is_hardlink 59478 .IP \[bu] 2 59479 Env Var: RCLONE_SFTP_COPY_IS_HARDLINK 59480 .IP \[bu] 2 59481 Type: bool 59482 .IP \[bu] 2 59483 Default: false 59484 .SS --sftp-description 59485 .PP 59486 Description of the remote 59487 .PP 59488 Properties: 59489 .IP \[bu] 2 59490 Config: description 59491 .IP \[bu] 2 59492 Env Var: RCLONE_SFTP_DESCRIPTION 59493 .IP \[bu] 2 59494 Type: string 59495 .IP \[bu] 2 59496 Required: false 59497 .SS Limitations 59498 .PP 59499 On some SFTP servers (e.g. 59500 Synology) the paths are different for SSH and SFTP so the hashes 59501 can\[aq]t be calculated properly. 59502 For them using \f[C]disable_hashcheck\f[R] is a good idea. 59503 .PP 59504 The only ssh agent supported under Windows is Putty\[aq]s pageant. 59505 .PP 59506 The Go SSH library disables the use of the aes128-cbc cipher by default, 59507 due to security concerns. 59508 This can be re-enabled on a per-connection basis by setting the 59509 \f[C]use_insecure_cipher\f[R] setting in the configuration file to 59510 \f[C]true\f[R]. 59511 Further details on the insecurity of this cipher can be found in this 59512 paper (http://www.isg.rhul.ac.uk/~kp/SandPfinal.pdf). 59513 .PP 59514 SFTP isn\[aq]t supported under plan9 until this 59515 issue (https://github.com/pkg/sftp/issues/156) is fixed. 59516 .PP 59517 Note that since SFTP isn\[aq]t HTTP based the following flags don\[aq]t 59518 work with it: \f[C]--dump-headers\f[R], \f[C]--dump-bodies\f[R], 59519 \f[C]--dump-auth\f[R]. 59520 .PP 59521 Note that \f[C]--timeout\f[R] and \f[C]--contimeout\f[R] are both 59522 supported. 59523 .SS rsync.net 59524 .PP 59525 rsync.net is supported through the SFTP backend. 59526 .PP 59527 See rsync.net\[aq]s documentation of rclone 59528 examples (https://www.rsync.net/products/rclone.html). 59529 .SS Hetzner Storage Box 59530 .PP 59531 Hetzner Storage Boxes are supported through the SFTP backend on port 23. 59532 .PP 59533 See Hetzner\[aq]s documentation for 59534 details (https://docs.hetzner.com/robot/storage-box/access/access-ssh-rsync-borg#rclone) 59535 .SH SMB 59536 .PP 59537 SMB is a communication protocol to share files over 59538 network (https://en.wikipedia.org/wiki/Server_Message_Block). 59539 .PP 59540 This relies on go-smb2 59541 library (https://github.com/hirochachacha/go-smb2/) for communication 59542 with SMB protocol. 59543 .PP 59544 Paths are specified as \f[C]remote:sharename\f[R] (or \f[C]remote:\f[R] 59545 for the \f[C]lsd\f[R] command.) You may put subdirectories in too, e.g. 59546 \f[C]remote:item/path/to/dir\f[R]. 59547 .SS Notes 59548 .PP 59549 The first path segment must be the name of the share, which you entered 59550 when you started to share on Windows. 59551 On smbd, it\[aq]s the section title in \f[C]smb.conf\f[R] (usually in 59552 \f[C]/etc/samba/\f[R]) file. 59553 You can find shares by querying the root if you\[aq]re unsure (e.g. 59554 \f[C]rclone lsd remote:\f[R]). 59555 .PP 59556 You can\[aq]t access to the shared printers from rclone, obviously. 59557 .PP 59558 You can\[aq]t use Anonymous access for logging in. 59559 You have to use the \f[C]guest\f[R] user with an empty password instead. 59560 The rclone client tries to avoid 8.3 names when uploading files by 59561 encoding trailing spaces and periods. 59562 Alternatively, the local 59563 backend (https://rclone.org/local/#paths-on-windows) on Windows can 59564 access SMB servers using UNC paths, by 59565 \f[C]\[rs]\[rs]server\[rs]share\f[R]. 59566 This doesn\[aq]t apply to non-Windows OSes, such as Linux and macOS. 59567 .SS Configuration 59568 .PP 59569 Here is an example of making a SMB configuration. 59570 .PP 59571 First run 59572 .IP 59573 .nf 59574 \f[C] 59575 rclone config 59576 \f[R] 59577 .fi 59578 .PP 59579 This will guide you through an interactive setup process. 59580 .IP 59581 .nf 59582 \f[C] 59583 No remotes found, make a new one? 59584 n) New remote 59585 s) Set configuration password 59586 q) Quit config 59587 n/s/q> n 59588 name> remote 59589 Option Storage. 59590 Type of storage to configure. 59591 Choose a number from below, or type in your own value. 59592 XX / SMB / CIFS 59593 \[rs] (smb) 59594 Storage> smb 59595 59596 Option host. 59597 Samba hostname to connect to. 59598 E.g. \[dq]example.com\[dq]. 59599 Enter a value. 59600 host> localhost 59601 59602 Option user. 59603 Samba username. 59604 Enter a string value. Press Enter for the default (lesmi). 59605 user> guest 59606 59607 Option port. 59608 Samba port number. 59609 Enter a signed integer. Press Enter for the default (445). 59610 port> 59611 59612 Option pass. 59613 Samba password. 59614 Choose an alternative below. Press Enter for the default (n). 59615 y) Yes, type in my own password 59616 g) Generate random password 59617 n) No, leave this optional password blank (default) 59618 y/g/n> g 59619 Password strength in bits. 59620 64 is just about memorable 59621 128 is secure 59622 1024 is the maximum 59623 Bits> 64 59624 Your password is: XXXX 59625 Use this password? Please note that an obscured version of this 59626 password (and not the password itself) will be stored under your 59627 configuration file, so keep this generated password in a safe place. 59628 y) Yes (default) 59629 n) No 59630 y/n> y 59631 59632 Option domain. 59633 Domain name for NTLM authentication. 59634 Enter a string value. Press Enter for the default (WORKGROUP). 59635 domain> 59636 59637 Edit advanced config? 59638 y) Yes 59639 n) No (default) 59640 y/n> n 59641 59642 Configuration complete. 59643 Options: 59644 - type: samba 59645 - host: localhost 59646 - user: guest 59647 - pass: *** ENCRYPTED *** 59648 Keep this \[dq]remote\[dq] remote? 59649 y) Yes this is OK (default) 59650 e) Edit this remote 59651 d) Delete this remote 59652 y/e/d> d 59653 \f[R] 59654 .fi 59655 .SS Standard options 59656 .PP 59657 Here are the Standard options specific to smb (SMB / CIFS). 59658 .SS --smb-host 59659 .PP 59660 SMB server hostname to connect to. 59661 .PP 59662 E.g. 59663 \[dq]example.com\[dq]. 59664 .PP 59665 Properties: 59666 .IP \[bu] 2 59667 Config: host 59668 .IP \[bu] 2 59669 Env Var: RCLONE_SMB_HOST 59670 .IP \[bu] 2 59671 Type: string 59672 .IP \[bu] 2 59673 Required: true 59674 .SS --smb-user 59675 .PP 59676 SMB username. 59677 .PP 59678 Properties: 59679 .IP \[bu] 2 59680 Config: user 59681 .IP \[bu] 2 59682 Env Var: RCLONE_SMB_USER 59683 .IP \[bu] 2 59684 Type: string 59685 .IP \[bu] 2 59686 Default: \[dq]$USER\[dq] 59687 .SS --smb-port 59688 .PP 59689 SMB port number. 59690 .PP 59691 Properties: 59692 .IP \[bu] 2 59693 Config: port 59694 .IP \[bu] 2 59695 Env Var: RCLONE_SMB_PORT 59696 .IP \[bu] 2 59697 Type: int 59698 .IP \[bu] 2 59699 Default: 445 59700 .SS --smb-pass 59701 .PP 59702 SMB password. 59703 .PP 59704 \f[B]NB\f[R] Input to this must be obscured - see rclone 59705 obscure (https://rclone.org/commands/rclone_obscure/). 59706 .PP 59707 Properties: 59708 .IP \[bu] 2 59709 Config: pass 59710 .IP \[bu] 2 59711 Env Var: RCLONE_SMB_PASS 59712 .IP \[bu] 2 59713 Type: string 59714 .IP \[bu] 2 59715 Required: false 59716 .SS --smb-domain 59717 .PP 59718 Domain name for NTLM authentication. 59719 .PP 59720 Properties: 59721 .IP \[bu] 2 59722 Config: domain 59723 .IP \[bu] 2 59724 Env Var: RCLONE_SMB_DOMAIN 59725 .IP \[bu] 2 59726 Type: string 59727 .IP \[bu] 2 59728 Default: \[dq]WORKGROUP\[dq] 59729 .SS --smb-spn 59730 .PP 59731 Service principal name. 59732 .PP 59733 Rclone presents this name to the server. 59734 Some servers use this as further authentication, and it often needs to 59735 be set for clusters. 59736 For example: 59737 .IP 59738 .nf 59739 \f[C] 59740 cifs/remotehost:1020 59741 \f[R] 59742 .fi 59743 .PP 59744 Leave blank if not sure. 59745 .PP 59746 Properties: 59747 .IP \[bu] 2 59748 Config: spn 59749 .IP \[bu] 2 59750 Env Var: RCLONE_SMB_SPN 59751 .IP \[bu] 2 59752 Type: string 59753 .IP \[bu] 2 59754 Required: false 59755 .SS Advanced options 59756 .PP 59757 Here are the Advanced options specific to smb (SMB / CIFS). 59758 .SS --smb-idle-timeout 59759 .PP 59760 Max time before closing idle connections. 59761 .PP 59762 If no connections have been returned to the connection pool in the time 59763 given, rclone will empty the connection pool. 59764 .PP 59765 Set to 0 to keep connections indefinitely. 59766 .PP 59767 Properties: 59768 .IP \[bu] 2 59769 Config: idle_timeout 59770 .IP \[bu] 2 59771 Env Var: RCLONE_SMB_IDLE_TIMEOUT 59772 .IP \[bu] 2 59773 Type: Duration 59774 .IP \[bu] 2 59775 Default: 1m0s 59776 .SS --smb-hide-special-share 59777 .PP 59778 Hide special shares (e.g. 59779 print$) which users aren\[aq]t supposed to access. 59780 .PP 59781 Properties: 59782 .IP \[bu] 2 59783 Config: hide_special_share 59784 .IP \[bu] 2 59785 Env Var: RCLONE_SMB_HIDE_SPECIAL_SHARE 59786 .IP \[bu] 2 59787 Type: bool 59788 .IP \[bu] 2 59789 Default: true 59790 .SS --smb-case-insensitive 59791 .PP 59792 Whether the server is configured to be case-insensitive. 59793 .PP 59794 Always true on Windows shares. 59795 .PP 59796 Properties: 59797 .IP \[bu] 2 59798 Config: case_insensitive 59799 .IP \[bu] 2 59800 Env Var: RCLONE_SMB_CASE_INSENSITIVE 59801 .IP \[bu] 2 59802 Type: bool 59803 .IP \[bu] 2 59804 Default: true 59805 .SS --smb-encoding 59806 .PP 59807 The encoding for the backend. 59808 .PP 59809 See the encoding section in the 59810 overview (https://rclone.org/overview/#encoding) for more info. 59811 .PP 59812 Properties: 59813 .IP \[bu] 2 59814 Config: encoding 59815 .IP \[bu] 2 59816 Env Var: RCLONE_SMB_ENCODING 59817 .IP \[bu] 2 59818 Type: Encoding 59819 .IP \[bu] 2 59820 Default: 59821 Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Ctl,RightSpace,RightPeriod,InvalidUtf8,Dot 59822 .SS --smb-description 59823 .PP 59824 Description of the remote 59825 .PP 59826 Properties: 59827 .IP \[bu] 2 59828 Config: description 59829 .IP \[bu] 2 59830 Env Var: RCLONE_SMB_DESCRIPTION 59831 .IP \[bu] 2 59832 Type: string 59833 .IP \[bu] 2 59834 Required: false 59835 .SH Storj 59836 .PP 59837 Storj (https://storj.io) is an encrypted, secure, and cost-effective 59838 object storage service that enables you to store, back up, and archive 59839 large amounts of data in a decentralized manner. 59840 .SS Backend options 59841 .PP 59842 Storj can be used both with this native backend and with the s3 backend 59843 using the Storj S3 compatible gateway (https://rclone.org/s3/#storj) 59844 (shared or private). 59845 .PP 59846 Use this backend to take advantage of client-side encryption as well as 59847 to achieve the best possible download performance. 59848 Uploads will be erasure-coded locally, thus a 1gb upload will result in 59849 2.68gb of data being uploaded to storage nodes across the network. 59850 .PP 59851 Use the s3 backend and one of the S3 compatible Hosted Gateways to 59852 increase upload performance and reduce the load on your systems and 59853 network. 59854 Uploads will be encrypted and erasure-coded server-side, thus a 1GB 59855 upload will result in only in 1GB of data being uploaded to storage 59856 nodes across the network. 59857 .PP 59858 Side by side comparison with more details: 59859 .IP \[bu] 2 59860 Characteristics: 59861 .RS 2 59862 .IP \[bu] 2 59863 \f[I]Storj backend\f[R]: Uses native RPC protocol, connects directly to 59864 the storage nodes which hosts the data. 59865 Requires more CPU resource of encoding/decoding and has network 59866 amplification (especially during the upload), uses lots of TCP 59867 connections 59868 .IP \[bu] 2 59869 \f[I]S3 backend\f[R]: Uses S3 compatible HTTP Rest API via the shared 59870 gateways. 59871 There is no network amplification, but performance depends on the shared 59872 gateways and the secret encryption key is shared with the gateway. 59873 .RE 59874 .IP \[bu] 2 59875 Typical usage: 59876 .RS 2 59877 .IP \[bu] 2 59878 \f[I]Storj backend\f[R]: Server environments and desktops with enough 59879 resources, internet speed and connectivity - and applications where 59880 storjs client-side encryption is required. 59881 .IP \[bu] 2 59882 \f[I]S3 backend\f[R]: Desktops and similar with limited resources, 59883 internet speed or connectivity. 59884 .RE 59885 .IP \[bu] 2 59886 Security: 59887 .RS 2 59888 .IP \[bu] 2 59889 \f[I]Storj backend\f[R]: \f[B]strong\f[R]. 59890 Private encryption key doesn\[aq]t need to leave the local computer. 59891 .IP \[bu] 2 59892 \f[I]S3 backend\f[R]: \f[B]weaker\f[R]. 59893 Private encryption key is shared 59894 with (https://docs.storj.io/dcs/api-reference/s3-compatible-gateway#security-and-encryption) 59895 the authentication service of the hosted gateway, where it\[aq]s stored 59896 encrypted. 59897 It can be stronger when combining with the rclone crypt backend. 59898 .RE 59899 .IP \[bu] 2 59900 Bandwidth usage (upload): 59901 .RS 2 59902 .IP \[bu] 2 59903 \f[I]Storj backend\f[R]: \f[B]higher\f[R]. 59904 As data is erasure coded on the client side both the original data and 59905 the parities should be uploaded. 59906 About \[ti]2.7 times more data is required to be uploaded. 59907 Client may start to upload with even higher number of nodes (\[ti]3.7 59908 times more) and abandon/stop the slow uploads. 59909 .IP \[bu] 2 59910 \f[I]S3 backend\f[R]: \f[B]normal\f[R]. 59911 Only the raw data is uploaded, erasure coding happens on the gateway. 59912 .RE 59913 .IP \[bu] 2 59914 Bandwidth usage (download) 59915 .RS 2 59916 .IP \[bu] 2 59917 \f[I]Storj backend\f[R]: \f[B]almost normal\f[R]. 59918 Only the minimal number of data is required, but to avoid very slow data 59919 providers a few more sources are used and the slowest are ignored (max 59920 1.2x overhead). 59921 .IP \[bu] 2 59922 \f[I]S3 backend\f[R]: \f[B]normal\f[R]. 59923 Only the raw data is downloaded, erasure coding happens on the shared 59924 gateway. 59925 .RE 59926 .IP \[bu] 2 59927 CPU usage: 59928 .RS 2 59929 .IP \[bu] 2 59930 \f[I]Storj backend\f[R]: \f[B]higher\f[R], but more predictable. 59931 Erasure code and encryption/decryption happens locally which requires 59932 significant CPU usage. 59933 .IP \[bu] 2 59934 \f[I]S3 backend\f[R]: \f[B]less\f[R]. 59935 Erasure code and encryption/decryption happens on shared s3 gateways 59936 (and as is, it depends on the current load on the gateways) 59937 .RE 59938 .IP \[bu] 2 59939 TCP connection usage: 59940 .RS 2 59941 .IP \[bu] 2 59942 \f[I]Storj backend\f[R]: \f[B]high\f[R]. 59943 A direct connection is required to each of the Storj nodes resulting in 59944 110 connections on upload and 35 on download per 64 MB segment. 59945 Not all the connections are actively used (slow ones are pruned), but 59946 they are all opened. 59947 Adjusting the max open file 59948 limit (https://rclone.org/storj/#known-issues) may be required. 59949 .IP \[bu] 2 59950 \f[I]S3 backend\f[R]: \f[B]normal\f[R]. 59951 Only one connection per download/upload thread is required to the shared 59952 gateway. 59953 .RE 59954 .IP \[bu] 2 59955 Overall performance: 59956 .RS 2 59957 .IP \[bu] 2 59958 \f[I]Storj backend\f[R]: with enough resources (CPU and bandwidth) 59959 \f[I]storj\f[R] backend can provide even 2x better performance. 59960 Data is directly downloaded to / uploaded from to the client instead of 59961 the gateway. 59962 .IP \[bu] 2 59963 \f[I]S3 backend\f[R]: Can be faster on edge devices where CPU and 59964 network bandwidth is limited as the shared S3 compatible gateways take 59965 care about the encrypting/decryption and erasure coding and no 59966 download/upload amplification. 59967 .RE 59968 .IP \[bu] 2 59969 Decentralization: 59970 .RS 2 59971 .IP \[bu] 2 59972 \f[I]Storj backend\f[R]: \f[B]high\f[R]. 59973 Data is downloaded directly from the distributed cloud of storage 59974 providers. 59975 .IP \[bu] 2 59976 \f[I]S3 backend\f[R]: \f[B]low\f[R]. 59977 Requires a running S3 gateway (either self-hosted or Storj-hosted). 59978 .RE 59979 .IP \[bu] 2 59980 Limitations: 59981 .RS 2 59982 .IP \[bu] 2 59983 \f[I]Storj backend\f[R]: \f[C]rclone checksum\f[R] is not possible 59984 without download, as checksum metadata is not calculated during upload 59985 .IP \[bu] 2 59986 \f[I]S3 backend\f[R]: secret encryption key is shared with the gateway 59987 .RE 59988 .SS Configuration 59989 .PP 59990 To make a new Storj configuration you need one of the following: * 59991 Access Grant that someone else shared with you. 59992 * API 59993 Key (https://documentation.storj.io/getting-started/uploading-your-first-object/create-an-api-key) 59994 of a Storj project you are a member of. 59995 .PP 59996 Here is an example of how to make a remote called \f[C]remote\f[R]. 59997 First run: 59998 .IP 59999 .nf 60000 \f[C] 60001 rclone config 60002 \f[R] 60003 .fi 60004 .PP 60005 This will guide you through an interactive setup process: 60006 .SS Setup with access grant 60007 .IP 60008 .nf 60009 \f[C] 60010 No remotes found, make a new one? 60011 n) New remote 60012 s) Set configuration password 60013 q) Quit config 60014 n/s/q> n 60015 name> remote 60016 Type of storage to configure. 60017 Enter a string value. Press Enter for the default (\[dq]\[dq]). 60018 Choose a number from below, or type in your own value 60019 [snip] 60020 XX / Storj Decentralized Cloud Storage 60021 \[rs] \[dq]storj\[dq] 60022 [snip] 60023 Storage> storj 60024 ** See help for storj backend at: https://rclone.org/storj/ ** 60025 60026 Choose an authentication method. 60027 Enter a string value. Press Enter for the default (\[dq]existing\[dq]). 60028 Choose a number from below, or type in your own value 60029 1 / Use an existing access grant. 60030 \[rs] \[dq]existing\[dq] 60031 2 / Create a new access grant from satellite address, API key, and passphrase. 60032 \[rs] \[dq]new\[dq] 60033 provider> existing 60034 Access Grant. 60035 Enter a string value. Press Enter for the default (\[dq]\[dq]). 60036 access_grant> your-access-grant-received-by-someone-else 60037 Remote config 60038 -------------------- 60039 [remote] 60040 type = storj 60041 access_grant = your-access-grant-received-by-someone-else 60042 -------------------- 60043 y) Yes this is OK (default) 60044 e) Edit this remote 60045 d) Delete this remote 60046 y/e/d> y 60047 \f[R] 60048 .fi 60049 .SS Setup with API key and passphrase 60050 .IP 60051 .nf 60052 \f[C] 60053 No remotes found, make a new one? 60054 n) New remote 60055 s) Set configuration password 60056 q) Quit config 60057 n/s/q> n 60058 name> remote 60059 Type of storage to configure. 60060 Enter a string value. Press Enter for the default (\[dq]\[dq]). 60061 Choose a number from below, or type in your own value 60062 [snip] 60063 XX / Storj Decentralized Cloud Storage 60064 \[rs] \[dq]storj\[dq] 60065 [snip] 60066 Storage> storj 60067 ** See help for storj backend at: https://rclone.org/storj/ ** 60068 60069 Choose an authentication method. 60070 Enter a string value. Press Enter for the default (\[dq]existing\[dq]). 60071 Choose a number from below, or type in your own value 60072 1 / Use an existing access grant. 60073 \[rs] \[dq]existing\[dq] 60074 2 / Create a new access grant from satellite address, API key, and passphrase. 60075 \[rs] \[dq]new\[dq] 60076 provider> new 60077 Satellite Address. Custom satellite address should match the format: \[ga]<nodeid>\[at]<address>:<port>\[ga]. 60078 Enter a string value. Press Enter for the default (\[dq]us1.storj.io\[dq]). 60079 Choose a number from below, or type in your own value 60080 1 / US1 60081 \[rs] \[dq]us1.storj.io\[dq] 60082 2 / EU1 60083 \[rs] \[dq]eu1.storj.io\[dq] 60084 3 / AP1 60085 \[rs] \[dq]ap1.storj.io\[dq] 60086 satellite_address> 1 60087 API Key. 60088 Enter a string value. Press Enter for the default (\[dq]\[dq]). 60089 api_key> your-api-key-for-your-storj-project 60090 Encryption Passphrase. To access existing objects enter passphrase used for uploading. 60091 Enter a string value. Press Enter for the default (\[dq]\[dq]). 60092 passphrase> your-human-readable-encryption-passphrase 60093 Remote config 60094 -------------------- 60095 [remote] 60096 type = storj 60097 satellite_address = 12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S\[at]us1.storj.io:7777 60098 api_key = your-api-key-for-your-storj-project 60099 passphrase = your-human-readable-encryption-passphrase 60100 access_grant = the-access-grant-generated-from-the-api-key-and-passphrase 60101 -------------------- 60102 y) Yes this is OK (default) 60103 e) Edit this remote 60104 d) Delete this remote 60105 y/e/d> y 60106 \f[R] 60107 .fi 60108 .SS Standard options 60109 .PP 60110 Here are the Standard options specific to storj (Storj Decentralized 60111 Cloud Storage). 60112 .SS --storj-provider 60113 .PP 60114 Choose an authentication method. 60115 .PP 60116 Properties: 60117 .IP \[bu] 2 60118 Config: provider 60119 .IP \[bu] 2 60120 Env Var: RCLONE_STORJ_PROVIDER 60121 .IP \[bu] 2 60122 Type: string 60123 .IP \[bu] 2 60124 Default: \[dq]existing\[dq] 60125 .IP \[bu] 2 60126 Examples: 60127 .RS 2 60128 .IP \[bu] 2 60129 \[dq]existing\[dq] 60130 .RS 2 60131 .IP \[bu] 2 60132 Use an existing access grant. 60133 .RE 60134 .IP \[bu] 2 60135 \[dq]new\[dq] 60136 .RS 2 60137 .IP \[bu] 2 60138 Create a new access grant from satellite address, API key, and 60139 passphrase. 60140 .RE 60141 .RE 60142 .SS --storj-access-grant 60143 .PP 60144 Access grant. 60145 .PP 60146 Properties: 60147 .IP \[bu] 2 60148 Config: access_grant 60149 .IP \[bu] 2 60150 Env Var: RCLONE_STORJ_ACCESS_GRANT 60151 .IP \[bu] 2 60152 Provider: existing 60153 .IP \[bu] 2 60154 Type: string 60155 .IP \[bu] 2 60156 Required: false 60157 .SS --storj-satellite-address 60158 .PP 60159 Satellite address. 60160 .PP 60161 Custom satellite address should match the format: 60162 \f[C]<nodeid>\[at]<address>:<port>\f[R]. 60163 .PP 60164 Properties: 60165 .IP \[bu] 2 60166 Config: satellite_address 60167 .IP \[bu] 2 60168 Env Var: RCLONE_STORJ_SATELLITE_ADDRESS 60169 .IP \[bu] 2 60170 Provider: new 60171 .IP \[bu] 2 60172 Type: string 60173 .IP \[bu] 2 60174 Default: \[dq]us1.storj.io\[dq] 60175 .IP \[bu] 2 60176 Examples: 60177 .RS 2 60178 .IP \[bu] 2 60179 \[dq]us1.storj.io\[dq] 60180 .RS 2 60181 .IP \[bu] 2 60182 US1 60183 .RE 60184 .IP \[bu] 2 60185 \[dq]eu1.storj.io\[dq] 60186 .RS 2 60187 .IP \[bu] 2 60188 EU1 60189 .RE 60190 .IP \[bu] 2 60191 \[dq]ap1.storj.io\[dq] 60192 .RS 2 60193 .IP \[bu] 2 60194 AP1 60195 .RE 60196 .RE 60197 .SS --storj-api-key 60198 .PP 60199 API key. 60200 .PP 60201 Properties: 60202 .IP \[bu] 2 60203 Config: api_key 60204 .IP \[bu] 2 60205 Env Var: RCLONE_STORJ_API_KEY 60206 .IP \[bu] 2 60207 Provider: new 60208 .IP \[bu] 2 60209 Type: string 60210 .IP \[bu] 2 60211 Required: false 60212 .SS --storj-passphrase 60213 .PP 60214 Encryption passphrase. 60215 .PP 60216 To access existing objects enter passphrase used for uploading. 60217 .PP 60218 Properties: 60219 .IP \[bu] 2 60220 Config: passphrase 60221 .IP \[bu] 2 60222 Env Var: RCLONE_STORJ_PASSPHRASE 60223 .IP \[bu] 2 60224 Provider: new 60225 .IP \[bu] 2 60226 Type: string 60227 .IP \[bu] 2 60228 Required: false 60229 .SS Advanced options 60230 .PP 60231 Here are the Advanced options specific to storj (Storj Decentralized 60232 Cloud Storage). 60233 .SS --storj-description 60234 .PP 60235 Description of the remote 60236 .PP 60237 Properties: 60238 .IP \[bu] 2 60239 Config: description 60240 .IP \[bu] 2 60241 Env Var: RCLONE_STORJ_DESCRIPTION 60242 .IP \[bu] 2 60243 Type: string 60244 .IP \[bu] 2 60245 Required: false 60246 .SS Usage 60247 .PP 60248 Paths are specified as \f[C]remote:bucket\f[R] (or \f[C]remote:\f[R] for 60249 the \f[C]lsf\f[R] command.) You may put subdirectories in too, e.g. 60250 \f[C]remote:bucket/path/to/dir\f[R]. 60251 .PP 60252 Once configured you can then use \f[C]rclone\f[R] like this. 60253 .SS Create a new bucket 60254 .PP 60255 Use the \f[C]mkdir\f[R] command to create new bucket, e.g. 60256 \f[C]bucket\f[R]. 60257 .IP 60258 .nf 60259 \f[C] 60260 rclone mkdir remote:bucket 60261 \f[R] 60262 .fi 60263 .SS List all buckets 60264 .PP 60265 Use the \f[C]lsf\f[R] command to list all buckets. 60266 .IP 60267 .nf 60268 \f[C] 60269 rclone lsf remote: 60270 \f[R] 60271 .fi 60272 .PP 60273 Note the colon (\f[C]:\f[R]) character at the end of the command line. 60274 .SS Delete a bucket 60275 .PP 60276 Use the \f[C]rmdir\f[R] command to delete an empty bucket. 60277 .IP 60278 .nf 60279 \f[C] 60280 rclone rmdir remote:bucket 60281 \f[R] 60282 .fi 60283 .PP 60284 Use the \f[C]purge\f[R] command to delete a non-empty bucket with all 60285 its content. 60286 .IP 60287 .nf 60288 \f[C] 60289 rclone purge remote:bucket 60290 \f[R] 60291 .fi 60292 .SS Upload objects 60293 .PP 60294 Use the \f[C]copy\f[R] command to upload an object. 60295 .IP 60296 .nf 60297 \f[C] 60298 rclone copy --progress /home/local/directory/file.ext remote:bucket/path/to/dir/ 60299 \f[R] 60300 .fi 60301 .PP 60302 The \f[C]--progress\f[R] flag is for displaying progress information. 60303 Remove it if you don\[aq]t need this information. 60304 .PP 60305 Use a folder in the local path to upload all its objects. 60306 .IP 60307 .nf 60308 \f[C] 60309 rclone copy --progress /home/local/directory/ remote:bucket/path/to/dir/ 60310 \f[R] 60311 .fi 60312 .PP 60313 Only modified files will be copied. 60314 .SS List objects 60315 .PP 60316 Use the \f[C]ls\f[R] command to list recursively all objects in a 60317 bucket. 60318 .IP 60319 .nf 60320 \f[C] 60321 rclone ls remote:bucket 60322 \f[R] 60323 .fi 60324 .PP 60325 Add the folder to the remote path to list recursively all objects in 60326 this folder. 60327 .IP 60328 .nf 60329 \f[C] 60330 rclone ls remote:bucket/path/to/dir/ 60331 \f[R] 60332 .fi 60333 .PP 60334 Use the \f[C]lsf\f[R] command to list non-recursively all objects in a 60335 bucket or a folder. 60336 .IP 60337 .nf 60338 \f[C] 60339 rclone lsf remote:bucket/path/to/dir/ 60340 \f[R] 60341 .fi 60342 .SS Download objects 60343 .PP 60344 Use the \f[C]copy\f[R] command to download an object. 60345 .IP 60346 .nf 60347 \f[C] 60348 rclone copy --progress remote:bucket/path/to/dir/file.ext /home/local/directory/ 60349 \f[R] 60350 .fi 60351 .PP 60352 The \f[C]--progress\f[R] flag is for displaying progress information. 60353 Remove it if you don\[aq]t need this information. 60354 .PP 60355 Use a folder in the remote path to download all its objects. 60356 .IP 60357 .nf 60358 \f[C] 60359 rclone copy --progress remote:bucket/path/to/dir/ /home/local/directory/ 60360 \f[R] 60361 .fi 60362 .SS Delete objects 60363 .PP 60364 Use the \f[C]deletefile\f[R] command to delete a single object. 60365 .IP 60366 .nf 60367 \f[C] 60368 rclone deletefile remote:bucket/path/to/dir/file.ext 60369 \f[R] 60370 .fi 60371 .PP 60372 Use the \f[C]delete\f[R] command to delete all object in a folder. 60373 .IP 60374 .nf 60375 \f[C] 60376 rclone delete remote:bucket/path/to/dir/ 60377 \f[R] 60378 .fi 60379 .SS Print the total size of objects 60380 .PP 60381 Use the \f[C]size\f[R] command to print the total size of objects in a 60382 bucket or a folder. 60383 .IP 60384 .nf 60385 \f[C] 60386 rclone size remote:bucket/path/to/dir/ 60387 \f[R] 60388 .fi 60389 .SS Sync two Locations 60390 .PP 60391 Use the \f[C]sync\f[R] command to sync the source to the destination, 60392 changing the destination only, deleting any excess files. 60393 .IP 60394 .nf 60395 \f[C] 60396 rclone sync --interactive --progress /home/local/directory/ remote:bucket/path/to/dir/ 60397 \f[R] 60398 .fi 60399 .PP 60400 The \f[C]--progress\f[R] flag is for displaying progress information. 60401 Remove it if you don\[aq]t need this information. 60402 .PP 60403 Since this can cause data loss, test first with the \f[C]--dry-run\f[R] 60404 flag to see exactly what would be copied and deleted. 60405 .PP 60406 The sync can be done also from Storj to the local file system. 60407 .IP 60408 .nf 60409 \f[C] 60410 rclone sync --interactive --progress remote:bucket/path/to/dir/ /home/local/directory/ 60411 \f[R] 60412 .fi 60413 .PP 60414 Or between two Storj buckets. 60415 .IP 60416 .nf 60417 \f[C] 60418 rclone sync --interactive --progress remote-us:bucket/path/to/dir/ remote-europe:bucket/path/to/dir/ 60419 \f[R] 60420 .fi 60421 .PP 60422 Or even between another cloud storage and Storj. 60423 .IP 60424 .nf 60425 \f[C] 60426 rclone sync --interactive --progress s3:bucket/path/to/dir/ storj:bucket/path/to/dir/ 60427 \f[R] 60428 .fi 60429 .SS Limitations 60430 .PP 60431 \f[C]rclone about\f[R] is not supported by the rclone Storj backend. 60432 Backends without this capability cannot determine free space for an 60433 rclone mount or use policy \f[C]mfs\f[R] (most free space) as a member 60434 of an rclone union remote. 60435 .PP 60436 See List of backends that do not support rclone 60437 about (https://rclone.org/overview/#optional-features) and rclone 60438 about (https://rclone.org/commands/rclone_about/) 60439 .SS Known issues 60440 .PP 60441 If you get errors like \f[C]too many open files\f[R] this usually 60442 happens when the default \f[C]ulimit\f[R] for system max open files is 60443 exceeded. 60444 Native Storj protocol opens a large number of TCP connections (each of 60445 which is counted as an open file). 60446 For a single upload stream you can expect 110 TCP connections to be 60447 opened. 60448 For a single download stream you can expect 35. 60449 This batch of connections will be opened for every 64 MiB segment and 60450 you should also expect TCP connections to be reused. 60451 If you do many transfers you eventually open a connection to most 60452 storage nodes (thousands of nodes). 60453 .PP 60454 To fix these, please raise your system limits. 60455 You can do this issuing a \f[C]ulimit -n 65536\f[R] just before you run 60456 rclone. 60457 To change the limits more permanently you can add this to your shell 60458 startup script, e.g. 60459 \f[C]$HOME/.bashrc\f[R], or change the system-wide configuration, 60460 usually \f[C]/etc/sysctl.conf\f[R] and/or 60461 \f[C]/etc/security/limits.conf\f[R], but please refer to your operating 60462 system manual. 60463 .SH SugarSync 60464 .PP 60465 SugarSync (https://sugarsync.com) is a cloud service that enables active 60466 synchronization of files across computers and other devices for file 60467 backup, access, syncing, and sharing. 60468 .SS Configuration 60469 .PP 60470 The initial setup for SugarSync involves getting a token from SugarSync 60471 which you can do with rclone. 60472 \f[C]rclone config\f[R] walks you through it. 60473 .PP 60474 Here is an example of how to make a remote called \f[C]remote\f[R]. 60475 First run: 60476 .IP 60477 .nf 60478 \f[C] 60479 rclone config 60480 \f[R] 60481 .fi 60482 .PP 60483 This will guide you through an interactive setup process: 60484 .IP 60485 .nf 60486 \f[C] 60487 No remotes found, make a new one? 60488 n) New remote 60489 s) Set configuration password 60490 q) Quit config 60491 n/s/q> n 60492 name> remote 60493 Type of storage to configure. 60494 Enter a string value. Press Enter for the default (\[dq]\[dq]). 60495 Choose a number from below, or type in your own value 60496 [snip] 60497 XX / Sugarsync 60498 \[rs] \[dq]sugarsync\[dq] 60499 [snip] 60500 Storage> sugarsync 60501 ** See help for sugarsync backend at: https://rclone.org/sugarsync/ ** 60502 60503 Sugarsync App ID. 60504 Leave blank to use rclone\[aq]s. 60505 Enter a string value. Press Enter for the default (\[dq]\[dq]). 60506 app_id> 60507 Sugarsync Access Key ID. 60508 Leave blank to use rclone\[aq]s. 60509 Enter a string value. Press Enter for the default (\[dq]\[dq]). 60510 access_key_id> 60511 Sugarsync Private Access Key 60512 Leave blank to use rclone\[aq]s. 60513 Enter a string value. Press Enter for the default (\[dq]\[dq]). 60514 private_access_key> 60515 Permanently delete files if true 60516 otherwise put them in the deleted files. 60517 Enter a boolean value (true or false). Press Enter for the default (\[dq]false\[dq]). 60518 hard_delete> 60519 Edit advanced config? (y/n) 60520 y) Yes 60521 n) No (default) 60522 y/n> n 60523 Remote config 60524 Username (email address)> nick\[at]craig-wood.com 60525 Your Sugarsync password is only required during setup and will not be stored. 60526 password: 60527 -------------------- 60528 [remote] 60529 type = sugarsync 60530 refresh_token = https://api.sugarsync.com/app-authorization/XXXXXXXXXXXXXXXXXX 60531 -------------------- 60532 y) Yes this is OK (default) 60533 e) Edit this remote 60534 d) Delete this remote 60535 y/e/d> y 60536 \f[R] 60537 .fi 60538 .PP 60539 Note that the config asks for your email and password but doesn\[aq]t 60540 store them, it only uses them to get the initial token. 60541 .PP 60542 Once configured you can then use \f[C]rclone\f[R] like this, 60543 .PP 60544 List directories (sync folders) in top level of your SugarSync 60545 .IP 60546 .nf 60547 \f[C] 60548 rclone lsd remote: 60549 \f[R] 60550 .fi 60551 .PP 60552 List all the files in your SugarSync folder \[dq]Test\[dq] 60553 .IP 60554 .nf 60555 \f[C] 60556 rclone ls remote:Test 60557 \f[R] 60558 .fi 60559 .PP 60560 To copy a local directory to an SugarSync folder called backup 60561 .IP 60562 .nf 60563 \f[C] 60564 rclone copy /home/source remote:backup 60565 \f[R] 60566 .fi 60567 .PP 60568 Paths are specified as \f[C]remote:path\f[R] 60569 .PP 60570 Paths may be as deep as required, e.g. 60571 \f[C]remote:directory/subdirectory\f[R]. 60572 .PP 60573 \f[B]NB\f[R] you can\[aq]t create files in the top level folder you have 60574 to create a folder, which rclone will create as a \[dq]Sync Folder\[dq] 60575 with SugarSync. 60576 .SS Modification times and hashes 60577 .PP 60578 SugarSync does not support modification times or hashes, therefore 60579 syncing will default to \f[C]--size-only\f[R] checking. 60580 Note that using \f[C]--update\f[R] will work as rclone can read the time 60581 files were uploaded. 60582 .SS Restricted filename characters 60583 .PP 60584 SugarSync replaces the default restricted characters 60585 set (https://rclone.org/overview/#restricted-characters) except for DEL. 60586 .PP 60587 Invalid UTF-8 bytes will also be 60588 replaced (https://rclone.org/overview/#invalid-utf8), as they can\[aq]t 60589 be used in XML strings. 60590 .SS Deleting files 60591 .PP 60592 Deleted files will be moved to the \[dq]Deleted items\[dq] folder by 60593 default. 60594 .PP 60595 However you can supply the flag \f[C]--sugarsync-hard-delete\f[R] or set 60596 the config parameter \f[C]hard_delete = true\f[R] if you would like 60597 files to be deleted straight away. 60598 .SS Standard options 60599 .PP 60600 Here are the Standard options specific to sugarsync (Sugarsync). 60601 .SS --sugarsync-app-id 60602 .PP 60603 Sugarsync App ID. 60604 .PP 60605 Leave blank to use rclone\[aq]s. 60606 .PP 60607 Properties: 60608 .IP \[bu] 2 60609 Config: app_id 60610 .IP \[bu] 2 60611 Env Var: RCLONE_SUGARSYNC_APP_ID 60612 .IP \[bu] 2 60613 Type: string 60614 .IP \[bu] 2 60615 Required: false 60616 .SS --sugarsync-access-key-id 60617 .PP 60618 Sugarsync Access Key ID. 60619 .PP 60620 Leave blank to use rclone\[aq]s. 60621 .PP 60622 Properties: 60623 .IP \[bu] 2 60624 Config: access_key_id 60625 .IP \[bu] 2 60626 Env Var: RCLONE_SUGARSYNC_ACCESS_KEY_ID 60627 .IP \[bu] 2 60628 Type: string 60629 .IP \[bu] 2 60630 Required: false 60631 .SS --sugarsync-private-access-key 60632 .PP 60633 Sugarsync Private Access Key. 60634 .PP 60635 Leave blank to use rclone\[aq]s. 60636 .PP 60637 Properties: 60638 .IP \[bu] 2 60639 Config: private_access_key 60640 .IP \[bu] 2 60641 Env Var: RCLONE_SUGARSYNC_PRIVATE_ACCESS_KEY 60642 .IP \[bu] 2 60643 Type: string 60644 .IP \[bu] 2 60645 Required: false 60646 .SS --sugarsync-hard-delete 60647 .PP 60648 Permanently delete files if true otherwise put them in the deleted 60649 files. 60650 .PP 60651 Properties: 60652 .IP \[bu] 2 60653 Config: hard_delete 60654 .IP \[bu] 2 60655 Env Var: RCLONE_SUGARSYNC_HARD_DELETE 60656 .IP \[bu] 2 60657 Type: bool 60658 .IP \[bu] 2 60659 Default: false 60660 .SS Advanced options 60661 .PP 60662 Here are the Advanced options specific to sugarsync (Sugarsync). 60663 .SS --sugarsync-refresh-token 60664 .PP 60665 Sugarsync refresh token. 60666 .PP 60667 Leave blank normally, will be auto configured by rclone. 60668 .PP 60669 Properties: 60670 .IP \[bu] 2 60671 Config: refresh_token 60672 .IP \[bu] 2 60673 Env Var: RCLONE_SUGARSYNC_REFRESH_TOKEN 60674 .IP \[bu] 2 60675 Type: string 60676 .IP \[bu] 2 60677 Required: false 60678 .SS --sugarsync-authorization 60679 .PP 60680 Sugarsync authorization. 60681 .PP 60682 Leave blank normally, will be auto configured by rclone. 60683 .PP 60684 Properties: 60685 .IP \[bu] 2 60686 Config: authorization 60687 .IP \[bu] 2 60688 Env Var: RCLONE_SUGARSYNC_AUTHORIZATION 60689 .IP \[bu] 2 60690 Type: string 60691 .IP \[bu] 2 60692 Required: false 60693 .SS --sugarsync-authorization-expiry 60694 .PP 60695 Sugarsync authorization expiry. 60696 .PP 60697 Leave blank normally, will be auto configured by rclone. 60698 .PP 60699 Properties: 60700 .IP \[bu] 2 60701 Config: authorization_expiry 60702 .IP \[bu] 2 60703 Env Var: RCLONE_SUGARSYNC_AUTHORIZATION_EXPIRY 60704 .IP \[bu] 2 60705 Type: string 60706 .IP \[bu] 2 60707 Required: false 60708 .SS --sugarsync-user 60709 .PP 60710 Sugarsync user. 60711 .PP 60712 Leave blank normally, will be auto configured by rclone. 60713 .PP 60714 Properties: 60715 .IP \[bu] 2 60716 Config: user 60717 .IP \[bu] 2 60718 Env Var: RCLONE_SUGARSYNC_USER 60719 .IP \[bu] 2 60720 Type: string 60721 .IP \[bu] 2 60722 Required: false 60723 .SS --sugarsync-root-id 60724 .PP 60725 Sugarsync root id. 60726 .PP 60727 Leave blank normally, will be auto configured by rclone. 60728 .PP 60729 Properties: 60730 .IP \[bu] 2 60731 Config: root_id 60732 .IP \[bu] 2 60733 Env Var: RCLONE_SUGARSYNC_ROOT_ID 60734 .IP \[bu] 2 60735 Type: string 60736 .IP \[bu] 2 60737 Required: false 60738 .SS --sugarsync-deleted-id 60739 .PP 60740 Sugarsync deleted folder id. 60741 .PP 60742 Leave blank normally, will be auto configured by rclone. 60743 .PP 60744 Properties: 60745 .IP \[bu] 2 60746 Config: deleted_id 60747 .IP \[bu] 2 60748 Env Var: RCLONE_SUGARSYNC_DELETED_ID 60749 .IP \[bu] 2 60750 Type: string 60751 .IP \[bu] 2 60752 Required: false 60753 .SS --sugarsync-encoding 60754 .PP 60755 The encoding for the backend. 60756 .PP 60757 See the encoding section in the 60758 overview (https://rclone.org/overview/#encoding) for more info. 60759 .PP 60760 Properties: 60761 .IP \[bu] 2 60762 Config: encoding 60763 .IP \[bu] 2 60764 Env Var: RCLONE_SUGARSYNC_ENCODING 60765 .IP \[bu] 2 60766 Type: Encoding 60767 .IP \[bu] 2 60768 Default: Slash,Ctl,InvalidUtf8,Dot 60769 .SS --sugarsync-description 60770 .PP 60771 Description of the remote 60772 .PP 60773 Properties: 60774 .IP \[bu] 2 60775 Config: description 60776 .IP \[bu] 2 60777 Env Var: RCLONE_SUGARSYNC_DESCRIPTION 60778 .IP \[bu] 2 60779 Type: string 60780 .IP \[bu] 2 60781 Required: false 60782 .SS Limitations 60783 .PP 60784 \f[C]rclone about\f[R] is not supported by the SugarSync backend. 60785 Backends without this capability cannot determine free space for an 60786 rclone mount or use policy \f[C]mfs\f[R] (most free space) as a member 60787 of an rclone union remote. 60788 .PP 60789 See List of backends that do not support rclone 60790 about (https://rclone.org/overview/#optional-features) and rclone 60791 about (https://rclone.org/commands/rclone_about/) 60792 .SH Tardigrade 60793 .PP 60794 The Tardigrade backend has been renamed to be the Storj 60795 backend (https://rclone.org/storj/). 60796 Old configuration files will continue to work. 60797 .SH Uptobox 60798 .PP 60799 This is a Backend for Uptobox file storage service. 60800 Uptobox is closer to a one-click hoster than a traditional cloud storage 60801 provider and therefore not suitable for long term storage. 60802 .PP 60803 Paths are specified as \f[C]remote:path\f[R] 60804 .PP 60805 Paths may be as deep as required, e.g. 60806 \f[C]remote:directory/subdirectory\f[R]. 60807 .SS Configuration 60808 .PP 60809 To configure an Uptobox backend you\[aq]ll need your personal api token. 60810 You\[aq]ll find it in your account 60811 settings (https://uptobox.com/my_account) 60812 .PP 60813 Here is an example of how to make a remote called \f[C]remote\f[R] with 60814 the default setup. 60815 First run: 60816 .IP 60817 .nf 60818 \f[C] 60819 rclone config 60820 \f[R] 60821 .fi 60822 .PP 60823 This will guide you through an interactive setup process: 60824 .IP 60825 .nf 60826 \f[C] 60827 Current remotes: 60828 60829 Name Type 60830 ==== ==== 60831 TestUptobox uptobox 60832 60833 e) Edit existing remote 60834 n) New remote 60835 d) Delete remote 60836 r) Rename remote 60837 c) Copy remote 60838 s) Set configuration password 60839 q) Quit config 60840 e/n/d/r/c/s/q> n 60841 name> uptobox 60842 Type of storage to configure. 60843 Enter a string value. Press Enter for the default (\[dq]\[dq]). 60844 Choose a number from below, or type in your own value 60845 [...] 60846 37 / Uptobox 60847 \[rs] \[dq]uptobox\[dq] 60848 [...] 60849 Storage> uptobox 60850 ** See help for uptobox backend at: https://rclone.org/uptobox/ ** 60851 60852 Your API Key, get it from https://uptobox.com/my_account 60853 Enter a string value. Press Enter for the default (\[dq]\[dq]). 60854 api_key> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 60855 Edit advanced config? (y/n) 60856 y) Yes 60857 n) No (default) 60858 y/n> n 60859 Remote config 60860 -------------------- 60861 [uptobox] 60862 type = uptobox 60863 api_key = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 60864 -------------------- 60865 y) Yes this is OK (default) 60866 e) Edit this remote 60867 d) Delete this remote 60868 y/e/d> 60869 \f[R] 60870 .fi 60871 .PP 60872 Once configured you can then use \f[C]rclone\f[R] like this, 60873 .PP 60874 List directories in top level of your Uptobox 60875 .IP 60876 .nf 60877 \f[C] 60878 rclone lsd remote: 60879 \f[R] 60880 .fi 60881 .PP 60882 List all the files in your Uptobox 60883 .IP 60884 .nf 60885 \f[C] 60886 rclone ls remote: 60887 \f[R] 60888 .fi 60889 .PP 60890 To copy a local directory to an Uptobox directory called backup 60891 .IP 60892 .nf 60893 \f[C] 60894 rclone copy /home/source remote:backup 60895 \f[R] 60896 .fi 60897 .SS Modification times and hashes 60898 .PP 60899 Uptobox supports neither modified times nor checksums. 60900 All timestamps will read as that set by \f[C]--default-time\f[R]. 60901 .SS Restricted filename characters 60902 .PP 60903 In addition to the default restricted characters 60904 set (https://rclone.org/overview/#restricted-characters) the following 60905 characters are also replaced: 60906 .PP 60907 .TS 60908 tab(@); 60909 l c c. 60910 T{ 60911 Character 60912 T}@T{ 60913 Value 60914 T}@T{ 60915 Replacement 60916 T} 60917 _ 60918 T{ 60919 \[dq] 60920 T}@T{ 60921 0x22 60922 T}@T{ 60923 \[uFF02] 60924 T} 60925 T{ 60926 \[ga] 60927 T}@T{ 60928 0x41 60929 T}@T{ 60930 \[uFF40] 60931 T} 60932 .TE 60933 .PP 60934 Invalid UTF-8 bytes will also be 60935 replaced (https://rclone.org/overview/#invalid-utf8), as they can\[aq]t 60936 be used in XML strings. 60937 .SS Standard options 60938 .PP 60939 Here are the Standard options specific to uptobox (Uptobox). 60940 .SS --uptobox-access-token 60941 .PP 60942 Your access token. 60943 .PP 60944 Get it from https://uptobox.com/my_account. 60945 .PP 60946 Properties: 60947 .IP \[bu] 2 60948 Config: access_token 60949 .IP \[bu] 2 60950 Env Var: RCLONE_UPTOBOX_ACCESS_TOKEN 60951 .IP \[bu] 2 60952 Type: string 60953 .IP \[bu] 2 60954 Required: false 60955 .SS Advanced options 60956 .PP 60957 Here are the Advanced options specific to uptobox (Uptobox). 60958 .SS --uptobox-private 60959 .PP 60960 Set to make uploaded files private 60961 .PP 60962 Properties: 60963 .IP \[bu] 2 60964 Config: private 60965 .IP \[bu] 2 60966 Env Var: RCLONE_UPTOBOX_PRIVATE 60967 .IP \[bu] 2 60968 Type: bool 60969 .IP \[bu] 2 60970 Default: false 60971 .SS --uptobox-encoding 60972 .PP 60973 The encoding for the backend. 60974 .PP 60975 See the encoding section in the 60976 overview (https://rclone.org/overview/#encoding) for more info. 60977 .PP 60978 Properties: 60979 .IP \[bu] 2 60980 Config: encoding 60981 .IP \[bu] 2 60982 Env Var: RCLONE_UPTOBOX_ENCODING 60983 .IP \[bu] 2 60984 Type: Encoding 60985 .IP \[bu] 2 60986 Default: 60987 Slash,LtGt,DoubleQuote,BackQuote,Del,Ctl,LeftSpace,InvalidUtf8,Dot 60988 .SS --uptobox-description 60989 .PP 60990 Description of the remote 60991 .PP 60992 Properties: 60993 .IP \[bu] 2 60994 Config: description 60995 .IP \[bu] 2 60996 Env Var: RCLONE_UPTOBOX_DESCRIPTION 60997 .IP \[bu] 2 60998 Type: string 60999 .IP \[bu] 2 61000 Required: false 61001 .SS Limitations 61002 .PP 61003 Uptobox will delete inactive files that have not been accessed in 60 61004 days. 61005 .PP 61006 \f[C]rclone about\f[R] is not supported by this backend an overview of 61007 used space can however been seen in the uptobox web interface. 61008 .SH Union 61009 .PP 61010 The \f[C]union\f[R] backend joins several remotes together to make a 61011 single unified view of them. 61012 .PP 61013 During the initial setup with \f[C]rclone config\f[R] you will specify 61014 the upstream remotes as a space separated list. 61015 The upstream remotes can either be a local paths or other remotes. 61016 .PP 61017 The attributes \f[C]:ro\f[R], \f[C]:nc\f[R] and \f[C]:writeback\f[R] can 61018 be attached to the end of the remote to tag the remote as \f[B]read 61019 only\f[R], \f[B]no create\f[R] or \f[B]writeback\f[R], e.g. 61020 \f[C]remote:directory/subdirectory:ro\f[R] or 61021 \f[C]remote:directory/subdirectory:nc\f[R]. 61022 .IP \[bu] 2 61023 \f[C]:ro\f[R] means files will only be read from here and never written 61024 .IP \[bu] 2 61025 \f[C]:nc\f[R] means new files or directories won\[aq]t be created here 61026 .IP \[bu] 2 61027 \f[C]:writeback\f[R] means files found in different remotes will be 61028 written back here. 61029 See the writeback section for more info. 61030 .PP 61031 Subfolders can be used in upstream remotes. 61032 Assume a union remote named \f[C]backup\f[R] with the remotes 61033 \f[C]mydrive:private/backup\f[R]. 61034 Invoking \f[C]rclone mkdir backup:desktop\f[R] is exactly the same as 61035 invoking \f[C]rclone mkdir mydrive:private/backup/desktop\f[R]. 61036 .PP 61037 There is no special handling of paths containing \f[C]..\f[R] segments. 61038 Invoking \f[C]rclone mkdir backup:../desktop\f[R] is exactly the same as 61039 invoking \f[C]rclone mkdir mydrive:private/backup/../desktop\f[R]. 61040 .SS Configuration 61041 .PP 61042 Here is an example of how to make a union called \f[C]remote\f[R] for 61043 local folders. 61044 First run: 61045 .IP 61046 .nf 61047 \f[C] 61048 rclone config 61049 \f[R] 61050 .fi 61051 .PP 61052 This will guide you through an interactive setup process: 61053 .IP 61054 .nf 61055 \f[C] 61056 No remotes found, make a new one? 61057 n) New remote 61058 s) Set configuration password 61059 q) Quit config 61060 n/s/q> n 61061 name> remote 61062 Type of storage to configure. 61063 Choose a number from below, or type in your own value 61064 [snip] 61065 XX / Union merges the contents of several remotes 61066 \[rs] \[dq]union\[dq] 61067 [snip] 61068 Storage> union 61069 List of space separated upstreams. 61070 Can be \[aq]upstreama:test/dir upstreamb:\[aq], \[aq]\[rs]\[dq]upstreama:test/space:ro dir\[rs]\[dq] upstreamb:\[aq], etc. 61071 Enter a string value. Press Enter for the default (\[dq]\[dq]). 61072 upstreams> remote1:dir1 remote2:dir2 remote3:dir3 61073 Policy to choose upstream on ACTION class. 61074 Enter a string value. Press Enter for the default (\[dq]epall\[dq]). 61075 action_policy> 61076 Policy to choose upstream on CREATE class. 61077 Enter a string value. Press Enter for the default (\[dq]epmfs\[dq]). 61078 create_policy> 61079 Policy to choose upstream on SEARCH class. 61080 Enter a string value. Press Enter for the default (\[dq]ff\[dq]). 61081 search_policy> 61082 Cache time of usage and free space (in seconds). This option is only useful when a path preserving policy is used. 61083 Enter a signed integer. Press Enter for the default (\[dq]120\[dq]). 61084 cache_time> 61085 Remote config 61086 -------------------- 61087 [remote] 61088 type = union 61089 upstreams = remote1:dir1 remote2:dir2 remote3:dir3 61090 -------------------- 61091 y) Yes this is OK 61092 e) Edit this remote 61093 d) Delete this remote 61094 y/e/d> y 61095 Current remotes: 61096 61097 Name Type 61098 ==== ==== 61099 remote union 61100 61101 e) Edit existing remote 61102 n) New remote 61103 d) Delete remote 61104 r) Rename remote 61105 c) Copy remote 61106 s) Set configuration password 61107 q) Quit config 61108 e/n/d/r/c/s/q> q 61109 \f[R] 61110 .fi 61111 .PP 61112 Once configured you can then use \f[C]rclone\f[R] like this, 61113 .PP 61114 List directories in top level in \f[C]remote1:dir1\f[R], 61115 \f[C]remote2:dir2\f[R] and \f[C]remote3:dir3\f[R] 61116 .IP 61117 .nf 61118 \f[C] 61119 rclone lsd remote: 61120 \f[R] 61121 .fi 61122 .PP 61123 List all the files in \f[C]remote1:dir1\f[R], \f[C]remote2:dir2\f[R] and 61124 \f[C]remote3:dir3\f[R] 61125 .IP 61126 .nf 61127 \f[C] 61128 rclone ls remote: 61129 \f[R] 61130 .fi 61131 .PP 61132 Copy another local directory to the union directory called source, which 61133 will be placed into \f[C]remote3:dir3\f[R] 61134 .IP 61135 .nf 61136 \f[C] 61137 rclone copy C:\[rs]source remote:source 61138 \f[R] 61139 .fi 61140 .SS Behavior / Policies 61141 .PP 61142 The behavior of union backend is inspired by 61143 trapexit/mergerfs (https://github.com/trapexit/mergerfs). 61144 All functions are grouped into 3 categories: \f[B]action\f[R], 61145 \f[B]create\f[R] and \f[B]search\f[R]. 61146 These functions and categories can be assigned a policy which dictates 61147 what file or directory is chosen when performing that behavior. 61148 Any policy can be assigned to a function or category though some may not 61149 be very useful in practice. 61150 For instance: \f[B]rand\f[R] (random) may be useful for file creation 61151 (create) but could lead to very odd behavior if used for 61152 \f[C]delete\f[R] if there were more than one copy of the file. 61153 .SS Function / Category classifications 61154 .PP 61155 .TS 61156 tab(@); 61157 lw(5.8n) lw(15.0n) lw(49.2n). 61158 T{ 61159 Category 61160 T}@T{ 61161 Description 61162 T}@T{ 61163 Functions 61164 T} 61165 _ 61166 T{ 61167 action 61168 T}@T{ 61169 Writing Existing file 61170 T}@T{ 61171 move, rmdir, rmdirs, delete, purge and copy, sync (as destination when 61172 file exist) 61173 T} 61174 T{ 61175 create 61176 T}@T{ 61177 Create non-existing file 61178 T}@T{ 61179 copy, sync (as destination when file not exist) 61180 T} 61181 T{ 61182 search 61183 T}@T{ 61184 Reading and listing file 61185 T}@T{ 61186 ls, lsd, lsl, cat, md5sum, sha1sum and copy, sync (as source) 61187 T} 61188 T{ 61189 N/A 61190 T}@T{ 61191 T}@T{ 61192 size, about 61193 T} 61194 .TE 61195 .SS Path Preservation 61196 .PP 61197 Policies, as described below, are of two basic types. 61198 \f[C]path preserving\f[R] and \f[C]non-path preserving\f[R]. 61199 .PP 61200 All policies which start with \f[C]ep\f[R] (\f[B]epff\f[R], 61201 \f[B]eplfs\f[R], \f[B]eplus\f[R], \f[B]epmfs\f[R], \f[B]eprand\f[R]) are 61202 \f[C]path preserving\f[R]. 61203 \f[C]ep\f[R] stands for \f[C]existing path\f[R]. 61204 .PP 61205 A path preserving policy will only consider upstreams where the relative 61206 path being accessed already exists. 61207 .PP 61208 When using non-path preserving policies paths will be created in target 61209 upstreams as necessary. 61210 .SS Quota Relevant Policies 61211 .PP 61212 Some policies rely on quota information. 61213 These policies should be used only if your upstreams support the 61214 respective quota fields. 61215 .PP 61216 .TS 61217 tab(@); 61218 l l. 61219 T{ 61220 Policy 61221 T}@T{ 61222 Required Field 61223 T} 61224 _ 61225 T{ 61226 lfs, eplfs 61227 T}@T{ 61228 Free 61229 T} 61230 T{ 61231 mfs, epmfs 61232 T}@T{ 61233 Free 61234 T} 61235 T{ 61236 lus, eplus 61237 T}@T{ 61238 Used 61239 T} 61240 T{ 61241 lno, eplno 61242 T}@T{ 61243 Objects 61244 T} 61245 .TE 61246 .PP 61247 To check if your upstream supports the field, run 61248 \f[C]rclone about remote: [flags]\f[R] and see if the required field 61249 exists. 61250 .SS Filters 61251 .PP 61252 Policies basically search upstream remotes and create a list of files / 61253 paths for functions to work on. 61254 The policy is responsible for filtering and sorting. 61255 The policy type defines the sorting but filtering is mostly uniform as 61256 described below. 61257 .IP \[bu] 2 61258 No \f[B]search\f[R] policies filter. 61259 .IP \[bu] 2 61260 All \f[B]action\f[R] policies will filter out remotes which are tagged 61261 as \f[B]read-only\f[R]. 61262 .IP \[bu] 2 61263 All \f[B]create\f[R] policies will filter out remotes which are tagged 61264 \f[B]read-only\f[R] or \f[B]no-create\f[R]. 61265 .PP 61266 If all remotes are filtered an error will be returned. 61267 .SS Policy descriptions 61268 .PP 61269 The policies definition are inspired by 61270 trapexit/mergerfs (https://github.com/trapexit/mergerfs) but not exactly 61271 the same. 61272 Some policy definition could be different due to the much larger latency 61273 of remote file systems. 61274 .PP 61275 .TS 61276 tab(@); 61277 lw(16.2n) lw(53.8n). 61278 T{ 61279 Policy 61280 T}@T{ 61281 Description 61282 T} 61283 _ 61284 T{ 61285 all 61286 T}@T{ 61287 Search category: same as \f[B]epall\f[R]. 61288 Action category: same as \f[B]epall\f[R]. 61289 Create category: act on all upstreams. 61290 T} 61291 T{ 61292 epall (existing path, all) 61293 T}@T{ 61294 Search category: Given this order configured, act on the first one found 61295 where the relative path exists. 61296 Action category: apply to all found. 61297 Create category: act on all upstreams where the relative path exists. 61298 T} 61299 T{ 61300 epff (existing path, first found) 61301 T}@T{ 61302 Act on the first one found, by the time upstreams reply, where the 61303 relative path exists. 61304 T} 61305 T{ 61306 eplfs (existing path, least free space) 61307 T}@T{ 61308 Of all the upstreams on which the relative path exists choose the one 61309 with the least free space. 61310 T} 61311 T{ 61312 eplus (existing path, least used space) 61313 T}@T{ 61314 Of all the upstreams on which the relative path exists choose the one 61315 with the least used space. 61316 T} 61317 T{ 61318 eplno (existing path, least number of objects) 61319 T}@T{ 61320 Of all the upstreams on which the relative path exists choose the one 61321 with the least number of objects. 61322 T} 61323 T{ 61324 epmfs (existing path, most free space) 61325 T}@T{ 61326 Of all the upstreams on which the relative path exists choose the one 61327 with the most free space. 61328 T} 61329 T{ 61330 eprand (existing path, random) 61331 T}@T{ 61332 Calls \f[B]epall\f[R] and then randomizes. 61333 Returns only one upstream. 61334 T} 61335 T{ 61336 ff (first found) 61337 T}@T{ 61338 Search category: same as \f[B]epff\f[R]. 61339 Action category: same as \f[B]epff\f[R]. 61340 Create category: Act on the first one found by the time upstreams reply. 61341 T} 61342 T{ 61343 lfs (least free space) 61344 T}@T{ 61345 Search category: same as \f[B]eplfs\f[R]. 61346 Action category: same as \f[B]eplfs\f[R]. 61347 Create category: Pick the upstream with the least available free space. 61348 T} 61349 T{ 61350 lus (least used space) 61351 T}@T{ 61352 Search category: same as \f[B]eplus\f[R]. 61353 Action category: same as \f[B]eplus\f[R]. 61354 Create category: Pick the upstream with the least used space. 61355 T} 61356 T{ 61357 lno (least number of objects) 61358 T}@T{ 61359 Search category: same as \f[B]eplno\f[R]. 61360 Action category: same as \f[B]eplno\f[R]. 61361 Create category: Pick the upstream with the least number of objects. 61362 T} 61363 T{ 61364 mfs (most free space) 61365 T}@T{ 61366 Search category: same as \f[B]epmfs\f[R]. 61367 Action category: same as \f[B]epmfs\f[R]. 61368 Create category: Pick the upstream with the most available free space. 61369 T} 61370 T{ 61371 newest 61372 T}@T{ 61373 Pick the file / directory with the largest mtime. 61374 T} 61375 T{ 61376 rand (random) 61377 T}@T{ 61378 Calls \f[B]all\f[R] and then randomizes. 61379 Returns only one upstream. 61380 T} 61381 .TE 61382 .SS Writeback 61383 .PP 61384 The tag \f[C]:writeback\f[R] on an upstream remote can be used to make a 61385 simple cache system like this: 61386 .IP 61387 .nf 61388 \f[C] 61389 [union] 61390 type = union 61391 action_policy = all 61392 create_policy = all 61393 search_policy = ff 61394 upstreams = /local:writeback remote:dir 61395 \f[R] 61396 .fi 61397 .PP 61398 When files are opened for read, if the file is in \f[C]remote:dir\f[R] 61399 but not \f[C]/local\f[R] then rclone will copy the file entirely into 61400 \f[C]/local\f[R] before returning a reference to the file in 61401 \f[C]/local\f[R]. 61402 The copy will be done with the equivalent of \f[C]rclone copy\f[R] so 61403 will use \f[C]--multi-thread-streams\f[R] if configured. 61404 Any copies will be logged with an INFO log. 61405 .PP 61406 When files are written, they will be written to both 61407 \f[C]remote:dir\f[R] and \f[C]/local\f[R]. 61408 .PP 61409 As many remotes as desired can be added to \f[C]upstreams\f[R] but there 61410 should only be one \f[C]:writeback\f[R] tag. 61411 .PP 61412 Rclone does not manage the \f[C]:writeback\f[R] remote in any way other 61413 than writing files back to it. 61414 So if you need to expire old files or manage the size then you will have 61415 to do this yourself. 61416 .SS Standard options 61417 .PP 61418 Here are the Standard options specific to union (Union merges the 61419 contents of several upstream fs). 61420 .SS --union-upstreams 61421 .PP 61422 List of space separated upstreams. 61423 .PP 61424 Can be \[aq]upstreama:test/dir upstreamb:\[aq], 61425 \[aq]\[dq]upstreama:test/space:ro dir\[dq] upstreamb:\[aq], etc. 61426 .PP 61427 Properties: 61428 .IP \[bu] 2 61429 Config: upstreams 61430 .IP \[bu] 2 61431 Env Var: RCLONE_UNION_UPSTREAMS 61432 .IP \[bu] 2 61433 Type: string 61434 .IP \[bu] 2 61435 Required: true 61436 .SS --union-action-policy 61437 .PP 61438 Policy to choose upstream on ACTION category. 61439 .PP 61440 Properties: 61441 .IP \[bu] 2 61442 Config: action_policy 61443 .IP \[bu] 2 61444 Env Var: RCLONE_UNION_ACTION_POLICY 61445 .IP \[bu] 2 61446 Type: string 61447 .IP \[bu] 2 61448 Default: \[dq]epall\[dq] 61449 .SS --union-create-policy 61450 .PP 61451 Policy to choose upstream on CREATE category. 61452 .PP 61453 Properties: 61454 .IP \[bu] 2 61455 Config: create_policy 61456 .IP \[bu] 2 61457 Env Var: RCLONE_UNION_CREATE_POLICY 61458 .IP \[bu] 2 61459 Type: string 61460 .IP \[bu] 2 61461 Default: \[dq]epmfs\[dq] 61462 .SS --union-search-policy 61463 .PP 61464 Policy to choose upstream on SEARCH category. 61465 .PP 61466 Properties: 61467 .IP \[bu] 2 61468 Config: search_policy 61469 .IP \[bu] 2 61470 Env Var: RCLONE_UNION_SEARCH_POLICY 61471 .IP \[bu] 2 61472 Type: string 61473 .IP \[bu] 2 61474 Default: \[dq]ff\[dq] 61475 .SS --union-cache-time 61476 .PP 61477 Cache time of usage and free space (in seconds). 61478 .PP 61479 This option is only useful when a path preserving policy is used. 61480 .PP 61481 Properties: 61482 .IP \[bu] 2 61483 Config: cache_time 61484 .IP \[bu] 2 61485 Env Var: RCLONE_UNION_CACHE_TIME 61486 .IP \[bu] 2 61487 Type: int 61488 .IP \[bu] 2 61489 Default: 120 61490 .SS Advanced options 61491 .PP 61492 Here are the Advanced options specific to union (Union merges the 61493 contents of several upstream fs). 61494 .SS --union-min-free-space 61495 .PP 61496 Minimum viable free space for lfs/eplfs policies. 61497 .PP 61498 If a remote has less than this much free space then it won\[aq]t be 61499 considered for use in lfs or eplfs policies. 61500 .PP 61501 Properties: 61502 .IP \[bu] 2 61503 Config: min_free_space 61504 .IP \[bu] 2 61505 Env Var: RCLONE_UNION_MIN_FREE_SPACE 61506 .IP \[bu] 2 61507 Type: SizeSuffix 61508 .IP \[bu] 2 61509 Default: 1Gi 61510 .SS --union-description 61511 .PP 61512 Description of the remote 61513 .PP 61514 Properties: 61515 .IP \[bu] 2 61516 Config: description 61517 .IP \[bu] 2 61518 Env Var: RCLONE_UNION_DESCRIPTION 61519 .IP \[bu] 2 61520 Type: string 61521 .IP \[bu] 2 61522 Required: false 61523 .SS Metadata 61524 .PP 61525 Any metadata supported by the underlying remote is read and written. 61526 .PP 61527 See the metadata (https://rclone.org/docs/#metadata) docs for more info. 61528 .SH WebDAV 61529 .PP 61530 Paths are specified as \f[C]remote:path\f[R] 61531 .PP 61532 Paths may be as deep as required, e.g. 61533 \f[C]remote:directory/subdirectory\f[R]. 61534 .SS Configuration 61535 .PP 61536 To configure the WebDAV remote you will need to have a URL for it, and a 61537 username and password. 61538 If you know what kind of system you are connecting to then rclone can 61539 enable extra features. 61540 .PP 61541 Here is an example of how to make a remote called \f[C]remote\f[R]. 61542 First run: 61543 .IP 61544 .nf 61545 \f[C] 61546 rclone config 61547 \f[R] 61548 .fi 61549 .PP 61550 This will guide you through an interactive setup process: 61551 .IP 61552 .nf 61553 \f[C] 61554 No remotes found, make a new one? 61555 n) New remote 61556 s) Set configuration password 61557 q) Quit config 61558 n/s/q> n 61559 name> remote 61560 Type of storage to configure. 61561 Choose a number from below, or type in your own value 61562 [snip] 61563 XX / WebDAV 61564 \[rs] \[dq]webdav\[dq] 61565 [snip] 61566 Storage> webdav 61567 URL of http host to connect to 61568 Choose a number from below, or type in your own value 61569 1 / Connect to example.com 61570 \[rs] \[dq]https://example.com\[dq] 61571 url> https://example.com/remote.php/webdav/ 61572 Name of the WebDAV site/service/software you are using 61573 Choose a number from below, or type in your own value 61574 1 / Fastmail Files 61575 \[rs] (fastmail) 61576 2 / Nextcloud 61577 \[rs] (nextcloud) 61578 3 / Owncloud 61579 \[rs] (owncloud) 61580 4 / Sharepoint Online, authenticated by Microsoft account 61581 \[rs] (sharepoint) 61582 5 / Sharepoint with NTLM authentication, usually self-hosted or on-premises 61583 \[rs] (sharepoint-ntlm) 61584 6 / rclone WebDAV server to serve a remote over HTTP via the WebDAV protocol 61585 \[rs] (rclone) 61586 7 / Other site/service or software 61587 \[rs] (other) 61588 vendor> 2 61589 User name 61590 user> user 61591 Password. 61592 y) Yes type in my own password 61593 g) Generate random password 61594 n) No leave this optional password blank 61595 y/g/n> y 61596 Enter the password: 61597 password: 61598 Confirm the password: 61599 password: 61600 Bearer token instead of user/pass (e.g. a Macaroon) 61601 bearer_token> 61602 Remote config 61603 -------------------- 61604 [remote] 61605 type = webdav 61606 url = https://example.com/remote.php/webdav/ 61607 vendor = nextcloud 61608 user = user 61609 pass = *** ENCRYPTED *** 61610 bearer_token = 61611 -------------------- 61612 y) Yes this is OK 61613 e) Edit this remote 61614 d) Delete this remote 61615 y/e/d> y 61616 \f[R] 61617 .fi 61618 .PP 61619 Once configured you can then use \f[C]rclone\f[R] like this, 61620 .PP 61621 List directories in top level of your WebDAV 61622 .IP 61623 .nf 61624 \f[C] 61625 rclone lsd remote: 61626 \f[R] 61627 .fi 61628 .PP 61629 List all the files in your WebDAV 61630 .IP 61631 .nf 61632 \f[C] 61633 rclone ls remote: 61634 \f[R] 61635 .fi 61636 .PP 61637 To copy a local directory to an WebDAV directory called backup 61638 .IP 61639 .nf 61640 \f[C] 61641 rclone copy /home/source remote:backup 61642 \f[R] 61643 .fi 61644 .SS Modification times and hashes 61645 .PP 61646 Plain WebDAV does not support modified times. 61647 However when used with Fastmail Files, Owncloud or Nextcloud rclone will 61648 support modified times. 61649 .PP 61650 Likewise plain WebDAV does not support hashes, however when used with 61651 Fastmail Files, Owncloud or Nextcloud rclone will support SHA1 and MD5 61652 hashes. 61653 Depending on the exact version of Owncloud or Nextcloud hashes may 61654 appear on all objects, or only on objects which had a hash uploaded with 61655 them. 61656 .SS Standard options 61657 .PP 61658 Here are the Standard options specific to webdav (WebDAV). 61659 .SS --webdav-url 61660 .PP 61661 URL of http host to connect to. 61662 .PP 61663 E.g. 61664 https://example.com. 61665 .PP 61666 Properties: 61667 .IP \[bu] 2 61668 Config: url 61669 .IP \[bu] 2 61670 Env Var: RCLONE_WEBDAV_URL 61671 .IP \[bu] 2 61672 Type: string 61673 .IP \[bu] 2 61674 Required: true 61675 .SS --webdav-vendor 61676 .PP 61677 Name of the WebDAV site/service/software you are using. 61678 .PP 61679 Properties: 61680 .IP \[bu] 2 61681 Config: vendor 61682 .IP \[bu] 2 61683 Env Var: RCLONE_WEBDAV_VENDOR 61684 .IP \[bu] 2 61685 Type: string 61686 .IP \[bu] 2 61687 Required: false 61688 .IP \[bu] 2 61689 Examples: 61690 .RS 2 61691 .IP \[bu] 2 61692 \[dq]fastmail\[dq] 61693 .RS 2 61694 .IP \[bu] 2 61695 Fastmail Files 61696 .RE 61697 .IP \[bu] 2 61698 \[dq]nextcloud\[dq] 61699 .RS 2 61700 .IP \[bu] 2 61701 Nextcloud 61702 .RE 61703 .IP \[bu] 2 61704 \[dq]owncloud\[dq] 61705 .RS 2 61706 .IP \[bu] 2 61707 Owncloud 61708 .RE 61709 .IP \[bu] 2 61710 \[dq]sharepoint\[dq] 61711 .RS 2 61712 .IP \[bu] 2 61713 Sharepoint Online, authenticated by Microsoft account 61714 .RE 61715 .IP \[bu] 2 61716 \[dq]sharepoint-ntlm\[dq] 61717 .RS 2 61718 .IP \[bu] 2 61719 Sharepoint with NTLM authentication, usually self-hosted or on-premises 61720 .RE 61721 .IP \[bu] 2 61722 \[dq]rclone\[dq] 61723 .RS 2 61724 .IP \[bu] 2 61725 rclone WebDAV server to serve a remote over HTTP via the WebDAV protocol 61726 .RE 61727 .IP \[bu] 2 61728 \[dq]other\[dq] 61729 .RS 2 61730 .IP \[bu] 2 61731 Other site/service or software 61732 .RE 61733 .RE 61734 .SS --webdav-user 61735 .PP 61736 User name. 61737 .PP 61738 In case NTLM authentication is used, the username should be in the 61739 format \[aq]Domain\[aq]. 61740 .PP 61741 Properties: 61742 .IP \[bu] 2 61743 Config: user 61744 .IP \[bu] 2 61745 Env Var: RCLONE_WEBDAV_USER 61746 .IP \[bu] 2 61747 Type: string 61748 .IP \[bu] 2 61749 Required: false 61750 .SS --webdav-pass 61751 .PP 61752 Password. 61753 .PP 61754 \f[B]NB\f[R] Input to this must be obscured - see rclone 61755 obscure (https://rclone.org/commands/rclone_obscure/). 61756 .PP 61757 Properties: 61758 .IP \[bu] 2 61759 Config: pass 61760 .IP \[bu] 2 61761 Env Var: RCLONE_WEBDAV_PASS 61762 .IP \[bu] 2 61763 Type: string 61764 .IP \[bu] 2 61765 Required: false 61766 .SS --webdav-bearer-token 61767 .PP 61768 Bearer token instead of user/pass (e.g. 61769 a Macaroon). 61770 .PP 61771 Properties: 61772 .IP \[bu] 2 61773 Config: bearer_token 61774 .IP \[bu] 2 61775 Env Var: RCLONE_WEBDAV_BEARER_TOKEN 61776 .IP \[bu] 2 61777 Type: string 61778 .IP \[bu] 2 61779 Required: false 61780 .SS Advanced options 61781 .PP 61782 Here are the Advanced options specific to webdav (WebDAV). 61783 .SS --webdav-bearer-token-command 61784 .PP 61785 Command to run to get a bearer token. 61786 .PP 61787 Properties: 61788 .IP \[bu] 2 61789 Config: bearer_token_command 61790 .IP \[bu] 2 61791 Env Var: RCLONE_WEBDAV_BEARER_TOKEN_COMMAND 61792 .IP \[bu] 2 61793 Type: string 61794 .IP \[bu] 2 61795 Required: false 61796 .SS --webdav-encoding 61797 .PP 61798 The encoding for the backend. 61799 .PP 61800 See the encoding section in the 61801 overview (https://rclone.org/overview/#encoding) for more info. 61802 .PP 61803 Default encoding is 61804 Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,Hash,Percent,BackSlash,Del,Ctl,LeftSpace,LeftTilde,RightSpace,RightPeriod,InvalidUtf8 61805 for sharepoint-ntlm or identity otherwise. 61806 .PP 61807 Properties: 61808 .IP \[bu] 2 61809 Config: encoding 61810 .IP \[bu] 2 61811 Env Var: RCLONE_WEBDAV_ENCODING 61812 .IP \[bu] 2 61813 Type: string 61814 .IP \[bu] 2 61815 Required: false 61816 .SS --webdav-headers 61817 .PP 61818 Set HTTP headers for all transactions. 61819 .PP 61820 Use this to set additional HTTP headers for all transactions 61821 .PP 61822 The input format is comma separated list of key,value pairs. 61823 Standard CSV encoding (https://godoc.org/encoding/csv) may be used. 61824 .PP 61825 For example, to set a Cookie use \[aq]Cookie,name=value\[aq], or 61826 \[aq]\[dq]Cookie\[dq],\[dq]name=value\[dq]\[aq]. 61827 .PP 61828 You can set multiple headers, e.g. 61829 \[aq]\[dq]Cookie\[dq],\[dq]name=value\[dq],\[dq]Authorization\[dq],\[dq]xxx\[dq]\[aq]. 61830 .PP 61831 Properties: 61832 .IP \[bu] 2 61833 Config: headers 61834 .IP \[bu] 2 61835 Env Var: RCLONE_WEBDAV_HEADERS 61836 .IP \[bu] 2 61837 Type: CommaSepList 61838 .IP \[bu] 2 61839 Default: 61840 .SS --webdav-pacer-min-sleep 61841 .PP 61842 Minimum time to sleep between API calls. 61843 .PP 61844 Properties: 61845 .IP \[bu] 2 61846 Config: pacer_min_sleep 61847 .IP \[bu] 2 61848 Env Var: RCLONE_WEBDAV_PACER_MIN_SLEEP 61849 .IP \[bu] 2 61850 Type: Duration 61851 .IP \[bu] 2 61852 Default: 10ms 61853 .SS --webdav-nextcloud-chunk-size 61854 .PP 61855 Nextcloud upload chunk size. 61856 .PP 61857 We recommend configuring your NextCloud instance to increase the max 61858 chunk size to 1 GB for better upload performances. 61859 See 61860 https://docs.nextcloud.com/server/latest/admin_manual/configuration_files/big_file_upload_configuration.html#adjust-chunk-size-on-nextcloud-side 61861 .PP 61862 Set to 0 to disable chunked uploading. 61863 .PP 61864 Properties: 61865 .IP \[bu] 2 61866 Config: nextcloud_chunk_size 61867 .IP \[bu] 2 61868 Env Var: RCLONE_WEBDAV_NEXTCLOUD_CHUNK_SIZE 61869 .IP \[bu] 2 61870 Type: SizeSuffix 61871 .IP \[bu] 2 61872 Default: 10Mi 61873 .SS --webdav-owncloud-exclude-shares 61874 .PP 61875 Exclude ownCloud shares 61876 .PP 61877 Properties: 61878 .IP \[bu] 2 61879 Config: owncloud_exclude_shares 61880 .IP \[bu] 2 61881 Env Var: RCLONE_WEBDAV_OWNCLOUD_EXCLUDE_SHARES 61882 .IP \[bu] 2 61883 Type: bool 61884 .IP \[bu] 2 61885 Default: false 61886 .SS --webdav-description 61887 .PP 61888 Description of the remote 61889 .PP 61890 Properties: 61891 .IP \[bu] 2 61892 Config: description 61893 .IP \[bu] 2 61894 Env Var: RCLONE_WEBDAV_DESCRIPTION 61895 .IP \[bu] 2 61896 Type: string 61897 .IP \[bu] 2 61898 Required: false 61899 .SS Provider notes 61900 .PP 61901 See below for notes on specific providers. 61902 .SS Fastmail Files 61903 .PP 61904 Use \f[C]https://webdav.fastmail.com/\f[R] or a subdirectory as the URL, 61905 and your Fastmail email \f[C]username\[at]domain.tld\f[R] as the 61906 username. 61907 Follow this 61908 documentation (https://www.fastmail.help/hc/en-us/articles/360058752854-App-passwords) 61909 to create an app password with access to \f[C]Files (WebDAV)\f[R] and 61910 use this as the password. 61911 .PP 61912 Fastmail supports modified times using the \f[C]X-OC-Mtime\f[R] header. 61913 .SS Owncloud 61914 .PP 61915 Click on the settings cog in the bottom right of the page and this will 61916 show the WebDAV URL that rclone needs in the config step. 61917 It will look something like 61918 \f[C]https://example.com/remote.php/webdav/\f[R]. 61919 .PP 61920 Owncloud supports modified times using the \f[C]X-OC-Mtime\f[R] header. 61921 .SS Nextcloud 61922 .PP 61923 This is configured in an identical way to Owncloud. 61924 Note that Nextcloud initially did not support streaming of files 61925 (\f[C]rcat\f[R]) whereas Owncloud did, but 61926 this (https://github.com/nextcloud/nextcloud-snap/issues/365) seems to 61927 be fixed as of 2020-11-27 (tested with rclone v1.53.1 and Nextcloud 61928 Server v19). 61929 .SS Sharepoint Online 61930 .PP 61931 Rclone can be used with Sharepoint provided by OneDrive for Business or 61932 Office365 Education Accounts. 61933 This feature is only needed for a few of these Accounts, mostly 61934 Office365 Education ones. 61935 These accounts are sometimes not verified by the domain owner 61936 github#1975 (https://github.com/artpar/artpar/issues/1975) 61937 .PP 61938 This means that these accounts can\[aq]t be added using the official API 61939 (other Accounts should work with the \[dq]onedrive\[dq] option). 61940 However, it is possible to access them using webdav. 61941 .PP 61942 To use a sharepoint remote with rclone, add it like this: First, you 61943 need to get your remote\[aq]s URL: 61944 .IP \[bu] 2 61945 Go here (https://onedrive.live.com/about/en-us/signin/) to open your 61946 OneDrive or to sign in 61947 .IP \[bu] 2 61948 Now take a look at your address bar, the URL should look like this: 61949 \f[C]https://[YOUR-DOMAIN]-my.sharepoint.com/personal/[YOUR-EMAIL]/_layouts/15/onedrive.aspx\f[R] 61950 .PP 61951 You\[aq]ll only need this URL up to the email address. 61952 After that, you\[aq]ll most likely want to add \[dq]/Documents\[dq]. 61953 That subdirectory contains the actual data stored on your OneDrive. 61954 .PP 61955 Add the remote to rclone like this: Configure the \f[C]url\f[R] as 61956 \f[C]https://[YOUR-DOMAIN]-my.sharepoint.com/personal/[YOUR-EMAIL]/Documents\f[R] 61957 and use your normal account email and password for \f[C]user\f[R] and 61958 \f[C]pass\f[R]. 61959 If you have 2FA enabled, you have to generate an app password. 61960 Set the \f[C]vendor\f[R] to \f[C]sharepoint\f[R]. 61961 .PP 61962 Your config file should look like this: 61963 .IP 61964 .nf 61965 \f[C] 61966 [sharepoint] 61967 type = webdav 61968 url = https://[YOUR-DOMAIN]-my.sharepoint.com/personal/[YOUR-EMAIL]/Documents 61969 vendor = sharepoint 61970 user = YourEmailAddress 61971 pass = encryptedpassword 61972 \f[R] 61973 .fi 61974 .SS Sharepoint with NTLM Authentication 61975 .PP 61976 Use this option in case your (hosted) Sharepoint is not tied to OneDrive 61977 accounts and uses NTLM authentication. 61978 .PP 61979 To get the \f[C]url\f[R] configuration, similarly to the above, first 61980 navigate to the desired directory in your browser to get the URL, then 61981 strip everything after the name of the opened directory. 61982 .PP 61983 Example: If the URL is: 61984 https://example.sharepoint.com/sites/12345/Documents/Forms/AllItems.aspx 61985 .PP 61986 The configuration to use would be: 61987 https://example.sharepoint.com/sites/12345/Documents 61988 .PP 61989 Set the \f[C]vendor\f[R] to \f[C]sharepoint-ntlm\f[R]. 61990 .PP 61991 NTLM uses domain and user name combination for authentication, set 61992 \f[C]user\f[R] to \f[C]DOMAIN\[rs]username\f[R]. 61993 .PP 61994 Your config file should look like this: 61995 .IP 61996 .nf 61997 \f[C] 61998 [sharepoint] 61999 type = webdav 62000 url = https://[YOUR-DOMAIN]/some-path-to/Documents 62001 vendor = sharepoint-ntlm 62002 user = DOMAIN\[rs]user 62003 pass = encryptedpassword 62004 \f[R] 62005 .fi 62006 .SS Required Flags for SharePoint 62007 .PP 62008 As SharePoint does some special things with uploaded documents, you 62009 won\[aq]t be able to use the documents size or the documents hash to 62010 compare if a file has been changed since the upload / which file is 62011 newer. 62012 .PP 62013 For Rclone calls copying files (especially Office files such as .docx, 62014 \&.xlsx, etc.) from/to SharePoint (like copy, sync, etc.), you should 62015 append these flags to ensure Rclone uses the \[dq]Last Modified\[dq] 62016 datetime property to compare your documents: 62017 .IP 62018 .nf 62019 \f[C] 62020 --ignore-size --ignore-checksum --update 62021 \f[R] 62022 .fi 62023 .SS Rclone 62024 .PP 62025 Use this option if you are hosting remotes over WebDAV provided by 62026 rclone. 62027 Read rclone serve webdav for more details. 62028 .PP 62029 rclone serve supports modified times using the \f[C]X-OC-Mtime\f[R] 62030 header. 62031 .SS dCache 62032 .PP 62033 dCache is a storage system that supports many protocols and 62034 authentication/authorisation schemes. 62035 For WebDAV clients, it allows users to authenticate with username and 62036 password (BASIC), X.509, Kerberos, and various bearer tokens, including 62037 Macaroons (https://www.dcache.org/manuals/workshop-2017-05-29-Umea/000-Final/anupam_macaroons_v02.pdf) 62038 and OpenID-Connect (https://en.wikipedia.org/wiki/OpenID_Connect) access 62039 tokens. 62040 .PP 62041 Configure as normal using the \f[C]other\f[R] type. 62042 Don\[aq]t enter a username or password, instead enter your Macaroon as 62043 the \f[C]bearer_token\f[R]. 62044 .PP 62045 The config will end up looking something like this. 62046 .IP 62047 .nf 62048 \f[C] 62049 [dcache] 62050 type = webdav 62051 url = https://dcache... 62052 vendor = other 62053 user = 62054 pass = 62055 bearer_token = your-macaroon 62056 \f[R] 62057 .fi 62058 .PP 62059 There is a 62060 script (https://github.com/sara-nl/GridScripts/blob/master/get-macaroon) 62061 that obtains a Macaroon from a dCache WebDAV endpoint, and creates an 62062 rclone config file. 62063 .PP 62064 Macaroons may also be obtained from the dCacheView 62065 web-browser/JavaScript client that comes with dCache. 62066 .SS OpenID-Connect 62067 .PP 62068 dCache also supports authenticating with OpenID-Connect access tokens. 62069 OpenID-Connect is a protocol (based on OAuth 2.0) that allows services 62070 to identify users who have authenticated with some central service. 62071 .PP 62072 Support for OpenID-Connect in rclone is currently achieved using another 62073 software package called 62074 oidc-agent (https://github.com/indigo-dc/oidc-agent). 62075 This is a command-line tool that facilitates obtaining an access token. 62076 Once installed and configured, an access token is obtained by running 62077 the \f[C]oidc-token\f[R] command. 62078 The following example shows a (shortened) access token obtained from the 62079 \f[I]XDC\f[R] OIDC Provider. 62080 .IP 62081 .nf 62082 \f[C] 62083 paul\[at]celebrimbor:\[ti]$ oidc-token XDC 62084 eyJraWQ[...]QFXDt0 62085 paul\[at]celebrimbor:\[ti]$ 62086 \f[R] 62087 .fi 62088 .PP 62089 \f[B]Note\f[R] Before the \f[C]oidc-token\f[R] command will work, the 62090 refresh token must be loaded into the oidc agent. 62091 This is done with the \f[C]oidc-add\f[R] command (e.g., 62092 \f[C]oidc-add XDC\f[R]). 62093 This is typically done once per login session. 62094 Full details on this and how to register oidc-agent with your OIDC 62095 Provider are provided in the oidc-agent 62096 documentation (https://indigo-dc.gitbooks.io/oidc-agent/). 62097 .PP 62098 The rclone \f[C]bearer_token_command\f[R] configuration option is used 62099 to fetch the access token from oidc-agent. 62100 .PP 62101 Configure as a normal WebDAV endpoint, using the \[aq]other\[aq] vendor, 62102 leaving the username and password empty. 62103 When prompted, choose to edit the advanced config and enter the command 62104 to get a bearer token (e.g., \f[C]oidc-agent XDC\f[R]). 62105 .PP 62106 The following example config shows a WebDAV endpoint that uses 62107 oidc-agent to supply an access token from the \f[I]XDC\f[R] OIDC 62108 Provider. 62109 .IP 62110 .nf 62111 \f[C] 62112 [dcache] 62113 type = webdav 62114 url = https://dcache.example.org/ 62115 vendor = other 62116 bearer_token_command = oidc-token XDC 62117 \f[R] 62118 .fi 62119 .SH Yandex Disk 62120 .PP 62121 Yandex Disk (https://disk.yandex.com) is a cloud storage solution 62122 created by Yandex (https://yandex.com). 62123 .SS Configuration 62124 .PP 62125 Here is an example of making a yandex configuration. 62126 First run 62127 .IP 62128 .nf 62129 \f[C] 62130 rclone config 62131 \f[R] 62132 .fi 62133 .PP 62134 This will guide you through an interactive setup process: 62135 .IP 62136 .nf 62137 \f[C] 62138 No remotes found, make a new one? 62139 n) New remote 62140 s) Set configuration password 62141 n/s> n 62142 name> remote 62143 Type of storage to configure. 62144 Choose a number from below, or type in your own value 62145 [snip] 62146 XX / Yandex Disk 62147 \[rs] \[dq]yandex\[dq] 62148 [snip] 62149 Storage> yandex 62150 Yandex Client Id - leave blank normally. 62151 client_id> 62152 Yandex Client Secret - leave blank normally. 62153 client_secret> 62154 Remote config 62155 Use web browser to automatically authenticate rclone with remote? 62156 * Say Y if the machine running rclone has a web browser you can use 62157 * Say N if running rclone on a (remote) machine without web browser access 62158 If not sure try Y. If Y failed, try N. 62159 y) Yes 62160 n) No 62161 y/n> y 62162 If your browser doesn\[aq]t open automatically go to the following link: http://127.0.0.1:53682/auth 62163 Log in and authorize rclone for access 62164 Waiting for code... 62165 Got code 62166 -------------------- 62167 [remote] 62168 client_id = 62169 client_secret = 62170 token = {\[dq]access_token\[dq]:\[dq]xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\[dq],\[dq]token_type\[dq]:\[dq]OAuth\[dq],\[dq]expiry\[dq]:\[dq]2016-12-29T12:27:11.362788025Z\[dq]} 62171 -------------------- 62172 y) Yes this is OK 62173 e) Edit this remote 62174 d) Delete this remote 62175 y/e/d> y 62176 \f[R] 62177 .fi 62178 .PP 62179 See the remote setup docs (https://rclone.org/remote_setup/) for how to 62180 set it up on a machine with no Internet browser available. 62181 .PP 62182 Note that rclone runs a webserver on your local machine to collect the 62183 token as returned from Yandex Disk. 62184 This only runs from the moment it opens your browser to the moment you 62185 get back the verification code. 62186 This is on \f[C]http://127.0.0.1:53682/\f[R] and this it may require you 62187 to unblock it temporarily if you are running a host firewall. 62188 .PP 62189 Once configured you can then use \f[C]rclone\f[R] like this, 62190 .PP 62191 See top level directories 62192 .IP 62193 .nf 62194 \f[C] 62195 rclone lsd remote: 62196 \f[R] 62197 .fi 62198 .PP 62199 Make a new directory 62200 .IP 62201 .nf 62202 \f[C] 62203 rclone mkdir remote:directory 62204 \f[R] 62205 .fi 62206 .PP 62207 List the contents of a directory 62208 .IP 62209 .nf 62210 \f[C] 62211 rclone ls remote:directory 62212 \f[R] 62213 .fi 62214 .PP 62215 Sync \f[C]/home/local/directory\f[R] to the remote path, deleting any 62216 excess files in the path. 62217 .IP 62218 .nf 62219 \f[C] 62220 rclone sync --interactive /home/local/directory remote:directory 62221 \f[R] 62222 .fi 62223 .PP 62224 Yandex paths may be as deep as required, e.g. 62225 \f[C]remote:directory/subdirectory\f[R]. 62226 .SS Modification times and hashes 62227 .PP 62228 Modified times are supported and are stored accurate to 1 ns in custom 62229 metadata called \f[C]rclone_modified\f[R] in RFC3339 with nanoseconds 62230 format. 62231 .PP 62232 The MD5 hash algorithm is natively supported by Yandex Disk. 62233 .SS Emptying Trash 62234 .PP 62235 If you wish to empty your trash you can use the 62236 \f[C]rclone cleanup remote:\f[R] command which will permanently delete 62237 all your trashed files. 62238 This command does not take any path arguments. 62239 .SS Quota information 62240 .PP 62241 To view your current quota you can use the 62242 \f[C]rclone about remote:\f[R] command which will display your usage 62243 limit (quota) and the current usage. 62244 .SS Restricted filename characters 62245 .PP 62246 The default restricted characters 62247 set (https://rclone.org/overview/#restricted-characters) are replaced. 62248 .PP 62249 Invalid UTF-8 bytes will also be 62250 replaced (https://rclone.org/overview/#invalid-utf8), as they can\[aq]t 62251 be used in JSON strings. 62252 .SS Standard options 62253 .PP 62254 Here are the Standard options specific to yandex (Yandex Disk). 62255 .SS --yandex-client-id 62256 .PP 62257 OAuth Client Id. 62258 .PP 62259 Leave blank normally. 62260 .PP 62261 Properties: 62262 .IP \[bu] 2 62263 Config: client_id 62264 .IP \[bu] 2 62265 Env Var: RCLONE_YANDEX_CLIENT_ID 62266 .IP \[bu] 2 62267 Type: string 62268 .IP \[bu] 2 62269 Required: false 62270 .SS --yandex-client-secret 62271 .PP 62272 OAuth Client Secret. 62273 .PP 62274 Leave blank normally. 62275 .PP 62276 Properties: 62277 .IP \[bu] 2 62278 Config: client_secret 62279 .IP \[bu] 2 62280 Env Var: RCLONE_YANDEX_CLIENT_SECRET 62281 .IP \[bu] 2 62282 Type: string 62283 .IP \[bu] 2 62284 Required: false 62285 .SS Advanced options 62286 .PP 62287 Here are the Advanced options specific to yandex (Yandex Disk). 62288 .SS --yandex-token 62289 .PP 62290 OAuth Access Token as a JSON blob. 62291 .PP 62292 Properties: 62293 .IP \[bu] 2 62294 Config: token 62295 .IP \[bu] 2 62296 Env Var: RCLONE_YANDEX_TOKEN 62297 .IP \[bu] 2 62298 Type: string 62299 .IP \[bu] 2 62300 Required: false 62301 .SS --yandex-auth-url 62302 .PP 62303 Auth server URL. 62304 .PP 62305 Leave blank to use the provider defaults. 62306 .PP 62307 Properties: 62308 .IP \[bu] 2 62309 Config: auth_url 62310 .IP \[bu] 2 62311 Env Var: RCLONE_YANDEX_AUTH_URL 62312 .IP \[bu] 2 62313 Type: string 62314 .IP \[bu] 2 62315 Required: false 62316 .SS --yandex-token-url 62317 .PP 62318 Token server url. 62319 .PP 62320 Leave blank to use the provider defaults. 62321 .PP 62322 Properties: 62323 .IP \[bu] 2 62324 Config: token_url 62325 .IP \[bu] 2 62326 Env Var: RCLONE_YANDEX_TOKEN_URL 62327 .IP \[bu] 2 62328 Type: string 62329 .IP \[bu] 2 62330 Required: false 62331 .SS --yandex-hard-delete 62332 .PP 62333 Delete files permanently rather than putting them into the trash. 62334 .PP 62335 Properties: 62336 .IP \[bu] 2 62337 Config: hard_delete 62338 .IP \[bu] 2 62339 Env Var: RCLONE_YANDEX_HARD_DELETE 62340 .IP \[bu] 2 62341 Type: bool 62342 .IP \[bu] 2 62343 Default: false 62344 .SS --yandex-encoding 62345 .PP 62346 The encoding for the backend. 62347 .PP 62348 See the encoding section in the 62349 overview (https://rclone.org/overview/#encoding) for more info. 62350 .PP 62351 Properties: 62352 .IP \[bu] 2 62353 Config: encoding 62354 .IP \[bu] 2 62355 Env Var: RCLONE_YANDEX_ENCODING 62356 .IP \[bu] 2 62357 Type: Encoding 62358 .IP \[bu] 2 62359 Default: Slash,Del,Ctl,InvalidUtf8,Dot 62360 .SS --yandex-description 62361 .PP 62362 Description of the remote 62363 .PP 62364 Properties: 62365 .IP \[bu] 2 62366 Config: description 62367 .IP \[bu] 2 62368 Env Var: RCLONE_YANDEX_DESCRIPTION 62369 .IP \[bu] 2 62370 Type: string 62371 .IP \[bu] 2 62372 Required: false 62373 .SS Limitations 62374 .PP 62375 When uploading very large files (bigger than about 5 GiB) you will need 62376 to increase the \f[C]--timeout\f[R] parameter. 62377 This is because Yandex pauses (perhaps to calculate the MD5SUM for the 62378 entire file) before returning confirmation that the file has been 62379 uploaded. 62380 The default handling of timeouts in rclone is to assume a 5 minute pause 62381 is an error and close the connection - you\[aq]ll see 62382 \f[C]net/http: timeout awaiting response headers\f[R] errors in the logs 62383 if this is happening. 62384 Setting the timeout to twice the max size of file in GiB should be 62385 enough, so if you want to upload a 30 GiB file set a timeout of 62386 \f[C]2 * 30 = 60m\f[R], that is \f[C]--timeout 60m\f[R]. 62387 .PP 62388 Having a Yandex Mail account is mandatory to use the Yandex.Disk 62389 subscription. 62390 Token generation will work without a mail account, but Rclone won\[aq]t 62391 be able to complete any actions. 62392 .IP 62393 .nf 62394 \f[C] 62395 [403 - DiskUnsupportedUserAccountTypeError] User account type is not supported. 62396 \f[R] 62397 .fi 62398 .SH Zoho Workdrive 62399 .PP 62400 Zoho WorkDrive (https://www.zoho.com/workdrive/) is a cloud storage 62401 solution created by Zoho (https://zoho.com). 62402 .SS Configuration 62403 .PP 62404 Here is an example of making a zoho configuration. 62405 First run 62406 .IP 62407 .nf 62408 \f[C] 62409 rclone config 62410 \f[R] 62411 .fi 62412 .PP 62413 This will guide you through an interactive setup process: 62414 .IP 62415 .nf 62416 \f[C] 62417 No remotes found, make a new one? 62418 n) New remote 62419 s) Set configuration password 62420 n/s> n 62421 name> remote 62422 Type of storage to configure. 62423 Enter a string value. Press Enter for the default (\[dq]\[dq]). 62424 Choose a number from below, or type in your own value 62425 [snip] 62426 XX / Zoho 62427 \[rs] \[dq]zoho\[dq] 62428 [snip] 62429 Storage> zoho 62430 ** See help for zoho backend at: https://rclone.org/zoho/ ** 62431 62432 OAuth Client Id 62433 Leave blank normally. 62434 Enter a string value. Press Enter for the default (\[dq]\[dq]). 62435 client_id> 62436 OAuth Client Secret 62437 Leave blank normally. 62438 Enter a string value. Press Enter for the default (\[dq]\[dq]). 62439 client_secret> 62440 Edit advanced config? (y/n) 62441 y) Yes 62442 n) No (default) 62443 y/n> n 62444 Remote config 62445 Use web browser to automatically authenticate rclone with remote? 62446 * Say Y if the machine running rclone has a web browser you can use 62447 * Say N if running rclone on a (remote) machine without web browser access 62448 If not sure try Y. If Y failed, try N. 62449 y) Yes (default) 62450 n) No 62451 y/n> 62452 If your browser doesn\[aq]t open automatically go to the following link: http://127.0.0.1:53682/auth?state=LVn0IHzxej1ZkmQw31d0wQ 62453 Log in and authorize rclone for access 62454 Waiting for code... 62455 Got code 62456 Choose a number from below, or type in your own value 62457 1 / MyTeam 62458 \[rs] \[dq]4u28602177065ff22426787a6745dba8954eb\[dq] 62459 Enter a Team ID> 1 62460 Choose a number from below, or type in your own value 62461 1 / General 62462 \[rs] \[dq]4u2869d2aa6fca04f4f2f896b6539243b85b1\[dq] 62463 Enter a Workspace ID> 1 62464 -------------------- 62465 [remote] 62466 type = zoho 62467 token = {\[dq]access_token\[dq]:\[dq]xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\[dq],\[dq]token_type\[dq]:\[dq]Zoho-oauthtoken\[dq],\[dq]refresh_token\[dq]:\[dq]xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\[dq],\[dq]expiry\[dq]:\[dq]2020-10-12T00:54:52.370275223+02:00\[dq]} 62468 root_folder_id = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 62469 -------------------- 62470 y) Yes this is OK (default) 62471 e) Edit this remote 62472 d) Delete this remote 62473 y/e/d> 62474 \f[R] 62475 .fi 62476 .PP 62477 See the remote setup docs (https://rclone.org/remote_setup/) for how to 62478 set it up on a machine with no Internet browser available. 62479 .PP 62480 Rclone runs a webserver on your local computer to collect the 62481 authorization token from Zoho Workdrive. 62482 This is only from the moment your browser is opened until the token is 62483 returned. 62484 The webserver runs on \f[C]http://127.0.0.1:53682/\f[R]. 62485 If local port \f[C]53682\f[R] is protected by a firewall you may need to 62486 temporarily unblock the firewall to complete authorization. 62487 .PP 62488 Once configured you can then use \f[C]rclone\f[R] like this, 62489 .PP 62490 See top level directories 62491 .IP 62492 .nf 62493 \f[C] 62494 rclone lsd remote: 62495 \f[R] 62496 .fi 62497 .PP 62498 Make a new directory 62499 .IP 62500 .nf 62501 \f[C] 62502 rclone mkdir remote:directory 62503 \f[R] 62504 .fi 62505 .PP 62506 List the contents of a directory 62507 .IP 62508 .nf 62509 \f[C] 62510 rclone ls remote:directory 62511 \f[R] 62512 .fi 62513 .PP 62514 Sync \f[C]/home/local/directory\f[R] to the remote path, deleting any 62515 excess files in the path. 62516 .IP 62517 .nf 62518 \f[C] 62519 rclone sync --interactive /home/local/directory remote:directory 62520 \f[R] 62521 .fi 62522 .PP 62523 Zoho paths may be as deep as required, eg 62524 \f[C]remote:directory/subdirectory\f[R]. 62525 .SS Modification times and hashes 62526 .PP 62527 Modified times are currently not supported for Zoho Workdrive 62528 .PP 62529 No hash algorithms are supported. 62530 .SS Usage information 62531 .PP 62532 To view your current quota you can use the 62533 \f[C]rclone about remote:\f[R] command which will display your current 62534 usage. 62535 .SS Restricted filename characters 62536 .PP 62537 Only control characters and invalid UTF-8 are replaced. 62538 In addition most Unicode full-width characters are not supported at all 62539 and will be removed from filenames during upload. 62540 .SS Standard options 62541 .PP 62542 Here are the Standard options specific to zoho (Zoho). 62543 .SS --zoho-client-id 62544 .PP 62545 OAuth Client Id. 62546 .PP 62547 Leave blank normally. 62548 .PP 62549 Properties: 62550 .IP \[bu] 2 62551 Config: client_id 62552 .IP \[bu] 2 62553 Env Var: RCLONE_ZOHO_CLIENT_ID 62554 .IP \[bu] 2 62555 Type: string 62556 .IP \[bu] 2 62557 Required: false 62558 .SS --zoho-client-secret 62559 .PP 62560 OAuth Client Secret. 62561 .PP 62562 Leave blank normally. 62563 .PP 62564 Properties: 62565 .IP \[bu] 2 62566 Config: client_secret 62567 .IP \[bu] 2 62568 Env Var: RCLONE_ZOHO_CLIENT_SECRET 62569 .IP \[bu] 2 62570 Type: string 62571 .IP \[bu] 2 62572 Required: false 62573 .SS --zoho-region 62574 .PP 62575 Zoho region to connect to. 62576 .PP 62577 You\[aq]ll have to use the region your organization is registered in. 62578 If not sure use the same top level domain as you connect to in your 62579 browser. 62580 .PP 62581 Properties: 62582 .IP \[bu] 2 62583 Config: region 62584 .IP \[bu] 2 62585 Env Var: RCLONE_ZOHO_REGION 62586 .IP \[bu] 2 62587 Type: string 62588 .IP \[bu] 2 62589 Required: false 62590 .IP \[bu] 2 62591 Examples: 62592 .RS 2 62593 .IP \[bu] 2 62594 \[dq]com\[dq] 62595 .RS 2 62596 .IP \[bu] 2 62597 United states / Global 62598 .RE 62599 .IP \[bu] 2 62600 \[dq]eu\[dq] 62601 .RS 2 62602 .IP \[bu] 2 62603 Europe 62604 .RE 62605 .IP \[bu] 2 62606 \[dq]in\[dq] 62607 .RS 2 62608 .IP \[bu] 2 62609 India 62610 .RE 62611 .IP \[bu] 2 62612 \[dq]jp\[dq] 62613 .RS 2 62614 .IP \[bu] 2 62615 Japan 62616 .RE 62617 .IP \[bu] 2 62618 \[dq]com.cn\[dq] 62619 .RS 2 62620 .IP \[bu] 2 62621 China 62622 .RE 62623 .IP \[bu] 2 62624 \[dq]com.au\[dq] 62625 .RS 2 62626 .IP \[bu] 2 62627 Australia 62628 .RE 62629 .RE 62630 .SS Advanced options 62631 .PP 62632 Here are the Advanced options specific to zoho (Zoho). 62633 .SS --zoho-token 62634 .PP 62635 OAuth Access Token as a JSON blob. 62636 .PP 62637 Properties: 62638 .IP \[bu] 2 62639 Config: token 62640 .IP \[bu] 2 62641 Env Var: RCLONE_ZOHO_TOKEN 62642 .IP \[bu] 2 62643 Type: string 62644 .IP \[bu] 2 62645 Required: false 62646 .SS --zoho-auth-url 62647 .PP 62648 Auth server URL. 62649 .PP 62650 Leave blank to use the provider defaults. 62651 .PP 62652 Properties: 62653 .IP \[bu] 2 62654 Config: auth_url 62655 .IP \[bu] 2 62656 Env Var: RCLONE_ZOHO_AUTH_URL 62657 .IP \[bu] 2 62658 Type: string 62659 .IP \[bu] 2 62660 Required: false 62661 .SS --zoho-token-url 62662 .PP 62663 Token server url. 62664 .PP 62665 Leave blank to use the provider defaults. 62666 .PP 62667 Properties: 62668 .IP \[bu] 2 62669 Config: token_url 62670 .IP \[bu] 2 62671 Env Var: RCLONE_ZOHO_TOKEN_URL 62672 .IP \[bu] 2 62673 Type: string 62674 .IP \[bu] 2 62675 Required: false 62676 .SS --zoho-encoding 62677 .PP 62678 The encoding for the backend. 62679 .PP 62680 See the encoding section in the 62681 overview (https://rclone.org/overview/#encoding) for more info. 62682 .PP 62683 Properties: 62684 .IP \[bu] 2 62685 Config: encoding 62686 .IP \[bu] 2 62687 Env Var: RCLONE_ZOHO_ENCODING 62688 .IP \[bu] 2 62689 Type: Encoding 62690 .IP \[bu] 2 62691 Default: Del,Ctl,InvalidUtf8 62692 .SS --zoho-description 62693 .PP 62694 Description of the remote 62695 .PP 62696 Properties: 62697 .IP \[bu] 2 62698 Config: description 62699 .IP \[bu] 2 62700 Env Var: RCLONE_ZOHO_DESCRIPTION 62701 .IP \[bu] 2 62702 Type: string 62703 .IP \[bu] 2 62704 Required: false 62705 .SS Setting up your own client_id 62706 .PP 62707 For Zoho we advise you to set up your own client_id. 62708 To do so you have to complete the following steps. 62709 .IP "1." 3 62710 Log in to the Zoho API Console (https://api-console.zoho.com) 62711 .IP "2." 3 62712 Create a new client of type \[dq]Server-based Application\[dq]. 62713 The name and website don\[aq]t matter, but you must add the redirect URL 62714 \f[C]http://localhost:53682/\f[R]. 62715 .IP "3." 3 62716 Once the client is created, you can go to the settings tab and enable it 62717 in other regions. 62718 .PP 62719 The client id and client secret can now be used with rclone. 62720 .SH Local Filesystem 62721 .PP 62722 Local paths are specified as normal filesystem paths, e.g. 62723 \f[C]/path/to/wherever\f[R], so 62724 .IP 62725 .nf 62726 \f[C] 62727 rclone sync --interactive /home/source /tmp/destination 62728 \f[R] 62729 .fi 62730 .PP 62731 Will sync \f[C]/home/source\f[R] to \f[C]/tmp/destination\f[R]. 62732 .SS Configuration 62733 .PP 62734 For consistencies sake one can also configure a remote of type 62735 \f[C]local\f[R] in the config file, and access the local filesystem 62736 using rclone remote paths, e.g. 62737 \f[C]remote:path/to/wherever\f[R], but it is probably easier not to. 62738 .SS Modification times 62739 .PP 62740 Rclone reads and writes the modification times using an accuracy 62741 determined by the OS. 62742 Typically this is 1ns on Linux, 10 ns on Windows and 1 Second on OS X. 62743 .SS Filenames 62744 .PP 62745 Filenames should be encoded in UTF-8 on disk. 62746 This is the normal case for Windows and OS X. 62747 .PP 62748 There is a bit more uncertainty in the Linux world, but new 62749 distributions will have UTF-8 encoded files names. 62750 If you are using an old Linux filesystem with non UTF-8 file names (e.g. 62751 latin1) then you can use the \f[C]convmv\f[R] tool to convert the 62752 filesystem to UTF-8. 62753 This tool is available in most distributions\[aq] package managers. 62754 .PP 62755 If an invalid (non-UTF8) filename is read, the invalid characters will 62756 be replaced with a quoted representation of the invalid bytes. 62757 The name \f[C]gro\[rs]xdf\f[R] will be transferred as 62758 \f[C]gro\[u201B]DF\f[R]. 62759 \f[C]rclone\f[R] will emit a debug message in this case (use 62760 \f[C]-v\f[R] to see), e.g. 62761 .IP 62762 .nf 62763 \f[C] 62764 Local file system at .: Replacing invalid UTF-8 characters in \[dq]gro\[rs]xdf\[dq] 62765 \f[R] 62766 .fi 62767 .SS Restricted characters 62768 .PP 62769 With the local backend, restrictions on the characters that are usable 62770 in file or directory names depend on the operating system. 62771 To check what rclone will replace by default on your system, run 62772 \f[C]rclone help flags local-encoding\f[R]. 62773 .PP 62774 On non Windows platforms the following characters are replaced when 62775 handling file names. 62776 .PP 62777 .TS 62778 tab(@); 62779 l c c. 62780 T{ 62781 Character 62782 T}@T{ 62783 Value 62784 T}@T{ 62785 Replacement 62786 T} 62787 _ 62788 T{ 62789 NUL 62790 T}@T{ 62791 0x00 62792 T}@T{ 62793 \[u2400] 62794 T} 62795 T{ 62796 / 62797 T}@T{ 62798 0x2F 62799 T}@T{ 62800 \[uFF0F] 62801 T} 62802 .TE 62803 .PP 62804 When running on Windows the following characters are replaced. 62805 This list is based on the Windows file naming 62806 conventions (https://docs.microsoft.com/de-de/windows/desktop/FileIO/naming-a-file#naming-conventions). 62807 .PP 62808 .TS 62809 tab(@); 62810 l c c. 62811 T{ 62812 Character 62813 T}@T{ 62814 Value 62815 T}@T{ 62816 Replacement 62817 T} 62818 _ 62819 T{ 62820 NUL 62821 T}@T{ 62822 0x00 62823 T}@T{ 62824 \[u2400] 62825 T} 62826 T{ 62827 SOH 62828 T}@T{ 62829 0x01 62830 T}@T{ 62831 \[u2401] 62832 T} 62833 T{ 62834 STX 62835 T}@T{ 62836 0x02 62837 T}@T{ 62838 \[u2402] 62839 T} 62840 T{ 62841 ETX 62842 T}@T{ 62843 0x03 62844 T}@T{ 62845 \[u2403] 62846 T} 62847 T{ 62848 EOT 62849 T}@T{ 62850 0x04 62851 T}@T{ 62852 \[u2404] 62853 T} 62854 T{ 62855 ENQ 62856 T}@T{ 62857 0x05 62858 T}@T{ 62859 \[u2405] 62860 T} 62861 T{ 62862 ACK 62863 T}@T{ 62864 0x06 62865 T}@T{ 62866 \[u2406] 62867 T} 62868 T{ 62869 BEL 62870 T}@T{ 62871 0x07 62872 T}@T{ 62873 \[u2407] 62874 T} 62875 T{ 62876 BS 62877 T}@T{ 62878 0x08 62879 T}@T{ 62880 \[u2408] 62881 T} 62882 T{ 62883 HT 62884 T}@T{ 62885 0x09 62886 T}@T{ 62887 \[u2409] 62888 T} 62889 T{ 62890 LF 62891 T}@T{ 62892 0x0A 62893 T}@T{ 62894 \[u240A] 62895 T} 62896 T{ 62897 VT 62898 T}@T{ 62899 0x0B 62900 T}@T{ 62901 \[u240B] 62902 T} 62903 T{ 62904 FF 62905 T}@T{ 62906 0x0C 62907 T}@T{ 62908 \[u240C] 62909 T} 62910 T{ 62911 CR 62912 T}@T{ 62913 0x0D 62914 T}@T{ 62915 \[u240D] 62916 T} 62917 T{ 62918 SO 62919 T}@T{ 62920 0x0E 62921 T}@T{ 62922 \[u240E] 62923 T} 62924 T{ 62925 SI 62926 T}@T{ 62927 0x0F 62928 T}@T{ 62929 \[u240F] 62930 T} 62931 T{ 62932 DLE 62933 T}@T{ 62934 0x10 62935 T}@T{ 62936 \[u2410] 62937 T} 62938 T{ 62939 DC1 62940 T}@T{ 62941 0x11 62942 T}@T{ 62943 \[u2411] 62944 T} 62945 T{ 62946 DC2 62947 T}@T{ 62948 0x12 62949 T}@T{ 62950 \[u2412] 62951 T} 62952 T{ 62953 DC3 62954 T}@T{ 62955 0x13 62956 T}@T{ 62957 \[u2413] 62958 T} 62959 T{ 62960 DC4 62961 T}@T{ 62962 0x14 62963 T}@T{ 62964 \[u2414] 62965 T} 62966 T{ 62967 NAK 62968 T}@T{ 62969 0x15 62970 T}@T{ 62971 \[u2415] 62972 T} 62973 T{ 62974 SYN 62975 T}@T{ 62976 0x16 62977 T}@T{ 62978 \[u2416] 62979 T} 62980 T{ 62981 ETB 62982 T}@T{ 62983 0x17 62984 T}@T{ 62985 \[u2417] 62986 T} 62987 T{ 62988 CAN 62989 T}@T{ 62990 0x18 62991 T}@T{ 62992 \[u2418] 62993 T} 62994 T{ 62995 EM 62996 T}@T{ 62997 0x19 62998 T}@T{ 62999 \[u2419] 63000 T} 63001 T{ 63002 SUB 63003 T}@T{ 63004 0x1A 63005 T}@T{ 63006 \[u241A] 63007 T} 63008 T{ 63009 ESC 63010 T}@T{ 63011 0x1B 63012 T}@T{ 63013 \[u241B] 63014 T} 63015 T{ 63016 FS 63017 T}@T{ 63018 0x1C 63019 T}@T{ 63020 \[u241C] 63021 T} 63022 T{ 63023 GS 63024 T}@T{ 63025 0x1D 63026 T}@T{ 63027 \[u241D] 63028 T} 63029 T{ 63030 RS 63031 T}@T{ 63032 0x1E 63033 T}@T{ 63034 \[u241E] 63035 T} 63036 T{ 63037 US 63038 T}@T{ 63039 0x1F 63040 T}@T{ 63041 \[u241F] 63042 T} 63043 T{ 63044 / 63045 T}@T{ 63046 0x2F 63047 T}@T{ 63048 \[uFF0F] 63049 T} 63050 T{ 63051 \[dq] 63052 T}@T{ 63053 0x22 63054 T}@T{ 63055 \[uFF02] 63056 T} 63057 T{ 63058 * 63059 T}@T{ 63060 0x2A 63061 T}@T{ 63062 \[uFF0A] 63063 T} 63064 T{ 63065 : 63066 T}@T{ 63067 0x3A 63068 T}@T{ 63069 \[uFF1A] 63070 T} 63071 T{ 63072 < 63073 T}@T{ 63074 0x3C 63075 T}@T{ 63076 \[uFF1C] 63077 T} 63078 T{ 63079 > 63080 T}@T{ 63081 0x3E 63082 T}@T{ 63083 \[uFF1E] 63084 T} 63085 T{ 63086 ? 63087 T}@T{ 63088 0x3F 63089 T}@T{ 63090 \[uFF1F] 63091 T} 63092 T{ 63093 \[rs] 63094 T}@T{ 63095 0x5C 63096 T}@T{ 63097 \[uFF3C] 63098 T} 63099 T{ 63100 | 63101 T}@T{ 63102 0x7C 63103 T}@T{ 63104 \[uFF5C] 63105 T} 63106 .TE 63107 .PP 63108 File names on Windows can also not end with the following characters. 63109 These only get replaced if they are the last character in the name: 63110 .PP 63111 .TS 63112 tab(@); 63113 l c c. 63114 T{ 63115 Character 63116 T}@T{ 63117 Value 63118 T}@T{ 63119 Replacement 63120 T} 63121 _ 63122 T{ 63123 SP 63124 T}@T{ 63125 0x20 63126 T}@T{ 63127 \[u2420] 63128 T} 63129 T{ 63130 \&. 63131 T}@T{ 63132 0x2E 63133 T}@T{ 63134 \[uFF0E] 63135 T} 63136 .TE 63137 .PP 63138 Invalid UTF-8 bytes will also be 63139 replaced (https://rclone.org/overview/#invalid-utf8), as they can\[aq]t 63140 be converted to UTF-16. 63141 .SS Paths on Windows 63142 .PP 63143 On Windows there are many ways of specifying a path to a file system 63144 resource. 63145 Local paths can be absolute, like 63146 \f[C]C:\[rs]path\[rs]to\[rs]wherever\f[R], or relative, like 63147 \f[C]..\[rs]wherever\f[R]. 63148 Network paths in UNC format, \f[C]\[rs]\[rs]server\[rs]share\f[R], are 63149 also supported. 63150 Path separator can be either \f[C]\[rs]\f[R] (as in 63151 \f[C]C:\[rs]path\[rs]to\[rs]wherever\f[R]) or \f[C]/\f[R] (as in 63152 \f[C]C:/path/to/wherever\f[R]). 63153 Length of these paths are limited to 259 characters for files and 247 63154 characters for directories, but there is an alternative extended-length 63155 path format increasing the limit to (approximately) 32,767 characters. 63156 This format requires absolute paths and the use of prefix 63157 \f[C]\[rs]\[rs]?\[rs]\f[R], e.g. 63158 \f[C]\[rs]\[rs]?\[rs]D:\[rs]some\[rs]very\[rs]long\[rs]path\f[R]. 63159 For convenience rclone will automatically convert regular paths into the 63160 corresponding extended-length paths, so in most cases you do not have to 63161 worry about this (read more below). 63162 .PP 63163 Note that Windows supports using the same prefix 63164 \f[C]\[rs]\[rs]?\[rs]\f[R] to specify path to volumes identified by 63165 their GUID, e.g. 63166 \f[C]\[rs]\[rs]?\[rs]Volume{b75e2c83-0000-0000-0000-602f00000000}\[rs]some\[rs]path\f[R]. 63167 This is \f[I]not\f[R] supported in rclone, due to an 63168 issue (https://github.com/golang/go/issues/39785) in go. 63169 .SS Long paths 63170 .PP 63171 Rclone handles long paths automatically, by converting all paths to 63172 extended-length path 63173 format (https://docs.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation), 63174 which allows paths up to 32,767 characters. 63175 .PP 63176 This conversion will ensure paths are absolute and prefix them with the 63177 \f[C]\[rs]\[rs]?\[rs]\f[R]. 63178 This is why you will see that your paths, for instance 63179 \f[C].\[rs]files\f[R] is shown as path 63180 \f[C]\[rs]\[rs]?\[rs]C:\[rs]files\f[R] in the output, and 63181 \f[C]\[rs]\[rs]server\[rs]share\f[R] as 63182 \f[C]\[rs]\[rs]?\[rs]UNC\[rs]server\[rs]share\f[R]. 63183 .PP 63184 However, in rare cases this may cause problems with buggy file system 63185 drivers like EncFS (https://github.com/artpar/artpar/issues/261). 63186 To disable UNC conversion globally, add this to your 63187 \f[C].rclone.conf\f[R] file: 63188 .IP 63189 .nf 63190 \f[C] 63191 [local] 63192 nounc = true 63193 \f[R] 63194 .fi 63195 .PP 63196 If you want to selectively disable UNC, you can add it to a separate 63197 entry like this: 63198 .IP 63199 .nf 63200 \f[C] 63201 [nounc] 63202 type = local 63203 nounc = true 63204 \f[R] 63205 .fi 63206 .PP 63207 And use rclone like this: 63208 .PP 63209 \f[C]rclone copy c:\[rs]src nounc:z:\[rs]dst\f[R] 63210 .PP 63211 This will use UNC paths on \f[C]c:\[rs]src\f[R] but not on 63212 \f[C]z:\[rs]dst\f[R]. 63213 Of course this will cause problems if the absolute path length of a file 63214 exceeds 259 characters on z, so only use this option if you have to. 63215 .SS Symlinks / Junction points 63216 .PP 63217 Normally rclone will ignore symlinks or junction points (which behave 63218 like symlinks under Windows). 63219 .PP 63220 If you supply \f[C]--copy-links\f[R] or \f[C]-L\f[R] then rclone will 63221 follow the symlink and copy the pointed to file or directory. 63222 Note that this flag is incompatible with \f[C]--links\f[R] / 63223 \f[C]-l\f[R]. 63224 .PP 63225 This flag applies to all commands. 63226 .PP 63227 For example, supposing you have a directory structure like this 63228 .IP 63229 .nf 63230 \f[C] 63231 $ tree /tmp/a 63232 /tmp/a 63233 \[u251C]\[u2500]\[u2500] b -> ../b 63234 \[u251C]\[u2500]\[u2500] expected -> ../expected 63235 \[u251C]\[u2500]\[u2500] one 63236 \[u2514]\[u2500]\[u2500] two 63237 \[u2514]\[u2500]\[u2500] three 63238 \f[R] 63239 .fi 63240 .PP 63241 Then you can see the difference with and without the flag like this 63242 .IP 63243 .nf 63244 \f[C] 63245 $ rclone ls /tmp/a 63246 6 one 63247 6 two/three 63248 \f[R] 63249 .fi 63250 .PP 63251 and 63252 .IP 63253 .nf 63254 \f[C] 63255 $ rclone -L ls /tmp/a 63256 4174 expected 63257 6 one 63258 6 two/three 63259 6 b/two 63260 6 b/one 63261 \f[R] 63262 .fi 63263 .SS --links, -l 63264 .PP 63265 Normally rclone will ignore symlinks or junction points (which behave 63266 like symlinks under Windows). 63267 .PP 63268 If you supply this flag then rclone will copy symbolic links from the 63269 local storage, and store them as text files, with a 63270 \[aq].rclonelink\[aq] suffix in the remote storage. 63271 .PP 63272 The text file will contain the target of the symbolic link (see 63273 example). 63274 .PP 63275 This flag applies to all commands. 63276 .PP 63277 For example, supposing you have a directory structure like this 63278 .IP 63279 .nf 63280 \f[C] 63281 $ tree /tmp/a 63282 /tmp/a 63283 \[u251C]\[u2500]\[u2500] file1 -> ./file4 63284 \[u2514]\[u2500]\[u2500] file2 -> /home/user/file3 63285 \f[R] 63286 .fi 63287 .PP 63288 Copying the entire directory with \[aq]-l\[aq] 63289 .IP 63290 .nf 63291 \f[C] 63292 $ rclone copyto -l /tmp/a/file1 remote:/tmp/a/ 63293 \f[R] 63294 .fi 63295 .PP 63296 The remote files are created with a \[aq].rclonelink\[aq] suffix 63297 .IP 63298 .nf 63299 \f[C] 63300 $ rclone ls remote:/tmp/a 63301 5 file1.rclonelink 63302 14 file2.rclonelink 63303 \f[R] 63304 .fi 63305 .PP 63306 The remote files will contain the target of the symbolic links 63307 .IP 63308 .nf 63309 \f[C] 63310 $ rclone cat remote:/tmp/a/file1.rclonelink 63311 \&./file4 63312 63313 $ rclone cat remote:/tmp/a/file2.rclonelink 63314 /home/user/file3 63315 \f[R] 63316 .fi 63317 .PP 63318 Copying them back with \[aq]-l\[aq] 63319 .IP 63320 .nf 63321 \f[C] 63322 $ rclone copyto -l remote:/tmp/a/ /tmp/b/ 63323 63324 $ tree /tmp/b 63325 /tmp/b 63326 \[u251C]\[u2500]\[u2500] file1 -> ./file4 63327 \[u2514]\[u2500]\[u2500] file2 -> /home/user/file3 63328 \f[R] 63329 .fi 63330 .PP 63331 However, if copied back without \[aq]-l\[aq] 63332 .IP 63333 .nf 63334 \f[C] 63335 $ rclone copyto remote:/tmp/a/ /tmp/b/ 63336 63337 $ tree /tmp/b 63338 /tmp/b 63339 \[u251C]\[u2500]\[u2500] file1.rclonelink 63340 \[u2514]\[u2500]\[u2500] file2.rclonelink 63341 \f[R] 63342 .fi 63343 .PP 63344 Note that this flag is incompatible with \f[C]-copy-links\f[R] / 63345 \f[C]-L\f[R]. 63346 .SS Restricting filesystems with --one-file-system 63347 .PP 63348 Normally rclone will recurse through filesystems as mounted. 63349 .PP 63350 However if you set \f[C]--one-file-system\f[R] or \f[C]-x\f[R] this 63351 tells rclone to stay in the filesystem specified by the root and not to 63352 recurse into different file systems. 63353 .PP 63354 For example if you have a directory hierarchy like this 63355 .IP 63356 .nf 63357 \f[C] 63358 root 63359 \[u251C]\[u2500]\[u2500] disk1 - disk1 mounted on the root 63360 \[br]\ \ \[u2514]\[u2500]\[u2500] file3 - stored on disk1 63361 \[u251C]\[u2500]\[u2500] disk2 - disk2 mounted on the root 63362 \[br]\ \ \[u2514]\[u2500]\[u2500] file4 - stored on disk12 63363 \[u251C]\[u2500]\[u2500] file1 - stored on the root disk 63364 \[u2514]\[u2500]\[u2500] file2 - stored on the root disk 63365 \f[R] 63366 .fi 63367 .PP 63368 Using \f[C]rclone --one-file-system copy root remote:\f[R] will only 63369 copy \f[C]file1\f[R] and \f[C]file2\f[R]. 63370 Eg 63371 .IP 63372 .nf 63373 \f[C] 63374 $ rclone -q --one-file-system ls root 63375 0 file1 63376 0 file2 63377 \f[R] 63378 .fi 63379 .IP 63380 .nf 63381 \f[C] 63382 $ rclone -q ls root 63383 0 disk1/file3 63384 0 disk2/file4 63385 0 file1 63386 0 file2 63387 \f[R] 63388 .fi 63389 .PP 63390 \f[B]NB\f[R] Rclone (like most unix tools such as \f[C]du\f[R], 63391 \f[C]rsync\f[R] and \f[C]tar\f[R]) treats a bind mount to the same 63392 device as being on the same filesystem. 63393 .PP 63394 \f[B]NB\f[R] This flag is only available on Unix based systems. 63395 On systems where it isn\[aq]t supported (e.g. 63396 Windows) it will be ignored. 63397 .SS Advanced options 63398 .PP 63399 Here are the Advanced options specific to local (Local Disk). 63400 .SS --local-nounc 63401 .PP 63402 Disable UNC (long path names) conversion on Windows. 63403 .PP 63404 Properties: 63405 .IP \[bu] 2 63406 Config: nounc 63407 .IP \[bu] 2 63408 Env Var: RCLONE_LOCAL_NOUNC 63409 .IP \[bu] 2 63410 Type: bool 63411 .IP \[bu] 2 63412 Default: false 63413 .IP \[bu] 2 63414 Examples: 63415 .RS 2 63416 .IP \[bu] 2 63417 \[dq]true\[dq] 63418 .RS 2 63419 .IP \[bu] 2 63420 Disables long file names. 63421 .RE 63422 .RE 63423 .SS --copy-links / -L 63424 .PP 63425 Follow symlinks and copy the pointed to item. 63426 .PP 63427 Properties: 63428 .IP \[bu] 2 63429 Config: copy_links 63430 .IP \[bu] 2 63431 Env Var: RCLONE_LOCAL_COPY_LINKS 63432 .IP \[bu] 2 63433 Type: bool 63434 .IP \[bu] 2 63435 Default: false 63436 .SS --links / -l 63437 .PP 63438 Translate symlinks to/from regular files with a \[aq].rclonelink\[aq] 63439 extension. 63440 .PP 63441 Properties: 63442 .IP \[bu] 2 63443 Config: links 63444 .IP \[bu] 2 63445 Env Var: RCLONE_LOCAL_LINKS 63446 .IP \[bu] 2 63447 Type: bool 63448 .IP \[bu] 2 63449 Default: false 63450 .SS --skip-links 63451 .PP 63452 Don\[aq]t warn about skipped symlinks. 63453 .PP 63454 This flag disables warning messages on skipped symlinks or junction 63455 points, as you explicitly acknowledge that they should be skipped. 63456 .PP 63457 Properties: 63458 .IP \[bu] 2 63459 Config: skip_links 63460 .IP \[bu] 2 63461 Env Var: RCLONE_LOCAL_SKIP_LINKS 63462 .IP \[bu] 2 63463 Type: bool 63464 .IP \[bu] 2 63465 Default: false 63466 .SS --local-zero-size-links 63467 .PP 63468 Assume the Stat size of links is zero (and read them instead) 63469 (deprecated). 63470 .PP 63471 Rclone used to use the Stat size of links as the link size, but this 63472 fails in quite a few places: 63473 .IP \[bu] 2 63474 Windows 63475 .IP \[bu] 2 63476 On some virtual filesystems (such ash LucidLink) 63477 .IP \[bu] 2 63478 Android 63479 .PP 63480 So rclone now always reads the link. 63481 .PP 63482 Properties: 63483 .IP \[bu] 2 63484 Config: zero_size_links 63485 .IP \[bu] 2 63486 Env Var: RCLONE_LOCAL_ZERO_SIZE_LINKS 63487 .IP \[bu] 2 63488 Type: bool 63489 .IP \[bu] 2 63490 Default: false 63491 .SS --local-unicode-normalization 63492 .PP 63493 Apply unicode NFC normalization to paths and filenames. 63494 .PP 63495 This flag can be used to normalize file names into unicode NFC form that 63496 are read from the local filesystem. 63497 .PP 63498 Rclone does not normally touch the encoding of file names it reads from 63499 the file system. 63500 .PP 63501 This can be useful when using macOS as it normally provides decomposed 63502 (NFD) unicode which in some language (eg Korean) doesn\[aq]t display 63503 properly on some OSes. 63504 .PP 63505 Note that rclone compares filenames with unicode normalization in the 63506 sync routine so this flag shouldn\[aq]t normally be used. 63507 .PP 63508 Properties: 63509 .IP \[bu] 2 63510 Config: unicode_normalization 63511 .IP \[bu] 2 63512 Env Var: RCLONE_LOCAL_UNICODE_NORMALIZATION 63513 .IP \[bu] 2 63514 Type: bool 63515 .IP \[bu] 2 63516 Default: false 63517 .SS --local-no-check-updated 63518 .PP 63519 Don\[aq]t check to see if the files change during upload. 63520 .PP 63521 Normally rclone checks the size and modification time of files as they 63522 are being uploaded and aborts with a message which starts \[dq]can\[aq]t 63523 copy - source file is being updated\[dq] if the file changes during 63524 upload. 63525 .PP 63526 However on some file systems this modification time check may fail (e.g. 63527 Glusterfs #2206 (https://github.com/artpar/artpar/issues/2206)) so this 63528 check can be disabled with this flag. 63529 .PP 63530 If this flag is set, rclone will use its best efforts to transfer a file 63531 which is being updated. 63532 If the file is only having things appended to it (e.g. 63533 a log) then rclone will transfer the log file with the size it had the 63534 first time rclone saw it. 63535 .PP 63536 If the file is being modified throughout (not just appended to) then the 63537 transfer may fail with a hash check failure. 63538 .PP 63539 In detail, once the file has had stat() called on it for the first time 63540 we: 63541 .IP \[bu] 2 63542 Only transfer the size that stat gave 63543 .IP \[bu] 2 63544 Only checksum the size that stat gave 63545 .IP \[bu] 2 63546 Don\[aq]t update the stat info for the file 63547 .PP 63548 \f[B]NB\f[R] do not use this flag on a Windows Volume Shadow (VSS). 63549 For some unknown reason, files in a VSS sometimes show different sizes 63550 from the directory listing (where the initial stat value comes from on 63551 Windows) and when stat is called on them directly. 63552 Other copy tools always use the direct stat value and setting this flag 63553 will disable that. 63554 .PP 63555 Properties: 63556 .IP \[bu] 2 63557 Config: no_check_updated 63558 .IP \[bu] 2 63559 Env Var: RCLONE_LOCAL_NO_CHECK_UPDATED 63560 .IP \[bu] 2 63561 Type: bool 63562 .IP \[bu] 2 63563 Default: false 63564 .SS --one-file-system / -x 63565 .PP 63566 Don\[aq]t cross filesystem boundaries (unix/macOS only). 63567 .PP 63568 Properties: 63569 .IP \[bu] 2 63570 Config: one_file_system 63571 .IP \[bu] 2 63572 Env Var: RCLONE_LOCAL_ONE_FILE_SYSTEM 63573 .IP \[bu] 2 63574 Type: bool 63575 .IP \[bu] 2 63576 Default: false 63577 .SS --local-case-sensitive 63578 .PP 63579 Force the filesystem to report itself as case sensitive. 63580 .PP 63581 Normally the local backend declares itself as case insensitive on 63582 Windows/macOS and case sensitive for everything else. 63583 Use this flag to override the default choice. 63584 .PP 63585 Properties: 63586 .IP \[bu] 2 63587 Config: case_sensitive 63588 .IP \[bu] 2 63589 Env Var: RCLONE_LOCAL_CASE_SENSITIVE 63590 .IP \[bu] 2 63591 Type: bool 63592 .IP \[bu] 2 63593 Default: false 63594 .SS --local-case-insensitive 63595 .PP 63596 Force the filesystem to report itself as case insensitive. 63597 .PP 63598 Normally the local backend declares itself as case insensitive on 63599 Windows/macOS and case sensitive for everything else. 63600 Use this flag to override the default choice. 63601 .PP 63602 Properties: 63603 .IP \[bu] 2 63604 Config: case_insensitive 63605 .IP \[bu] 2 63606 Env Var: RCLONE_LOCAL_CASE_INSENSITIVE 63607 .IP \[bu] 2 63608 Type: bool 63609 .IP \[bu] 2 63610 Default: false 63611 .SS --local-no-preallocate 63612 .PP 63613 Disable preallocation of disk space for transferred files. 63614 .PP 63615 Preallocation of disk space helps prevent filesystem fragmentation. 63616 However, some virtual filesystem layers (such as Google Drive File 63617 Stream) may incorrectly set the actual file size equal to the 63618 preallocated space, causing checksum and file size checks to fail. 63619 Use this flag to disable preallocation. 63620 .PP 63621 Properties: 63622 .IP \[bu] 2 63623 Config: no_preallocate 63624 .IP \[bu] 2 63625 Env Var: RCLONE_LOCAL_NO_PREALLOCATE 63626 .IP \[bu] 2 63627 Type: bool 63628 .IP \[bu] 2 63629 Default: false 63630 .SS --local-no-sparse 63631 .PP 63632 Disable sparse files for multi-thread downloads. 63633 .PP 63634 On Windows platforms rclone will make sparse files when doing 63635 multi-thread downloads. 63636 This avoids long pauses on large files where the OS zeros the file. 63637 However sparse files may be undesirable as they cause disk fragmentation 63638 and can be slow to work with. 63639 .PP 63640 Properties: 63641 .IP \[bu] 2 63642 Config: no_sparse 63643 .IP \[bu] 2 63644 Env Var: RCLONE_LOCAL_NO_SPARSE 63645 .IP \[bu] 2 63646 Type: bool 63647 .IP \[bu] 2 63648 Default: false 63649 .SS --local-no-set-modtime 63650 .PP 63651 Disable setting modtime. 63652 .PP 63653 Normally rclone updates modification time of files after they are done 63654 uploading. 63655 This can cause permissions issues on Linux platforms when the user 63656 rclone is running as does not own the file uploaded, such as when 63657 copying to a CIFS mount owned by another user. 63658 If this option is enabled, rclone will no longer update the modtime 63659 after copying a file. 63660 .PP 63661 Properties: 63662 .IP \[bu] 2 63663 Config: no_set_modtime 63664 .IP \[bu] 2 63665 Env Var: RCLONE_LOCAL_NO_SET_MODTIME 63666 .IP \[bu] 2 63667 Type: bool 63668 .IP \[bu] 2 63669 Default: false 63670 .SS --local-encoding 63671 .PP 63672 The encoding for the backend. 63673 .PP 63674 See the encoding section in the 63675 overview (https://rclone.org/overview/#encoding) for more info. 63676 .PP 63677 Properties: 63678 .IP \[bu] 2 63679 Config: encoding 63680 .IP \[bu] 2 63681 Env Var: RCLONE_LOCAL_ENCODING 63682 .IP \[bu] 2 63683 Type: Encoding 63684 .IP \[bu] 2 63685 Default: Slash,Dot 63686 .SS --local-description 63687 .PP 63688 Description of the remote 63689 .PP 63690 Properties: 63691 .IP \[bu] 2 63692 Config: description 63693 .IP \[bu] 2 63694 Env Var: RCLONE_LOCAL_DESCRIPTION 63695 .IP \[bu] 2 63696 Type: string 63697 .IP \[bu] 2 63698 Required: false 63699 .SS Metadata 63700 .PP 63701 Depending on which OS is in use the local backend may return only some 63702 of the system metadata. 63703 Setting system metadata is supported on all OSes but setting user 63704 metadata is only supported on linux, freebsd, netbsd, macOS and Solaris. 63705 It is \f[B]not\f[R] supported on Windows yet (see 63706 pkg/attrs#47 (https://github.com/pkg/xattr/issues/47)). 63707 .PP 63708 User metadata is stored as extended attributes (which may not be 63709 supported by all file systems) under the \[dq]user.*\[dq] prefix. 63710 .PP 63711 Metadata is supported on files and directories. 63712 .PP 63713 Here are the possible system metadata items for the local backend. 63714 .PP 63715 .TS 63716 tab(@); 63717 lw(11.1n) lw(11.1n) lw(11.1n) lw(16.6n) lw(20.3n). 63718 T{ 63719 Name 63720 T}@T{ 63721 Help 63722 T}@T{ 63723 Type 63724 T}@T{ 63725 Example 63726 T}@T{ 63727 Read Only 63728 T} 63729 _ 63730 T{ 63731 atime 63732 T}@T{ 63733 Time of last access 63734 T}@T{ 63735 RFC 3339 63736 T}@T{ 63737 2006-01-02T15:04:05.999999999Z07:00 63738 T}@T{ 63739 N 63740 T} 63741 T{ 63742 btime 63743 T}@T{ 63744 Time of file birth (creation) 63745 T}@T{ 63746 RFC 3339 63747 T}@T{ 63748 2006-01-02T15:04:05.999999999Z07:00 63749 T}@T{ 63750 N 63751 T} 63752 T{ 63753 gid 63754 T}@T{ 63755 Group ID of owner 63756 T}@T{ 63757 decimal number 63758 T}@T{ 63759 500 63760 T}@T{ 63761 N 63762 T} 63763 T{ 63764 mode 63765 T}@T{ 63766 File type and mode 63767 T}@T{ 63768 octal, unix style 63769 T}@T{ 63770 0100664 63771 T}@T{ 63772 N 63773 T} 63774 T{ 63775 mtime 63776 T}@T{ 63777 Time of last modification 63778 T}@T{ 63779 RFC 3339 63780 T}@T{ 63781 2006-01-02T15:04:05.999999999Z07:00 63782 T}@T{ 63783 N 63784 T} 63785 T{ 63786 rdev 63787 T}@T{ 63788 Device ID (if special file) 63789 T}@T{ 63790 hexadecimal 63791 T}@T{ 63792 1abc 63793 T}@T{ 63794 N 63795 T} 63796 T{ 63797 uid 63798 T}@T{ 63799 User ID of owner 63800 T}@T{ 63801 decimal number 63802 T}@T{ 63803 500 63804 T}@T{ 63805 N 63806 T} 63807 .TE 63808 .PP 63809 See the metadata (https://rclone.org/docs/#metadata) docs for more info. 63810 .SS Backend commands 63811 .PP 63812 Here are the commands specific to the local backend. 63813 .PP 63814 Run them with 63815 .IP 63816 .nf 63817 \f[C] 63818 rclone backend COMMAND remote: 63819 \f[R] 63820 .fi 63821 .PP 63822 The help below will explain what arguments each command takes. 63823 .PP 63824 See the backend (https://rclone.org/commands/rclone_backend/) command 63825 for more info on how to pass options and arguments. 63826 .PP 63827 These can be run on a running backend using the rc command 63828 backend/command (https://rclone.org/rc/#backend-command). 63829 .SS noop 63830 .PP 63831 A null operation for testing backend commands 63832 .IP 63833 .nf 63834 \f[C] 63835 rclone backend noop remote: [options] [<arguments>+] 63836 \f[R] 63837 .fi 63838 .PP 63839 This is a test command which has some options you can try to change the 63840 output. 63841 .PP 63842 Options: 63843 .IP \[bu] 2 63844 \[dq]echo\[dq]: echo the input arguments 63845 .IP \[bu] 2 63846 \[dq]error\[dq]: return an error based on option value 63847 .SH Changelog 63848 .SS v1.66.0 - 2024-03-10 63849 .PP 63850 See commits (https://github.com/artpar/artpar/compare/v1.65.0...v1.66.0) 63851 .IP \[bu] 2 63852 Major features 63853 .RS 2 63854 .IP \[bu] 2 63855 Rclone will now sync directory modification times if the backend 63856 supports it. 63857 .RS 2 63858 .IP \[bu] 2 63859 This can be disabled with 63860 --no-update-dir-modtime (https://rclone.org/docs/#no-update-dir-modtime) 63861 .IP \[bu] 2 63862 See the overview (https://rclone.org/overview/#features) and look for 63863 the \f[C]D\f[R] flags in the \f[C]ModTime\f[R] column to see which 63864 backends support it. 63865 .RE 63866 .IP \[bu] 2 63867 Rclone will now sync directory metadata if the backend supports it when 63868 \f[C]-M\f[R]/\f[C]--metadata\f[R] is in use. 63869 .RS 2 63870 .IP \[bu] 2 63871 See the overview (https://rclone.org/overview/#features) and look for 63872 the \f[C]D\f[R] flags in the \f[C]Metadata\f[R] column to see which 63873 backends support it. 63874 .RE 63875 .IP \[bu] 2 63876 Bisync has received many updates see below for more details or 63877 bisync\[aq]s changelog (https://rclone.org/bisync/#changelog) 63878 .RE 63879 .IP \[bu] 2 63880 Removed backends 63881 .RS 2 63882 .IP \[bu] 2 63883 amazonclouddrive: Remove Amazon Drive backend code and docs (Nick 63884 Craig-Wood) 63885 .RE 63886 .IP \[bu] 2 63887 New Features 63888 .RS 2 63889 .IP \[bu] 2 63890 backend 63891 .RS 2 63892 .IP \[bu] 2 63893 Add description field for all backends (Paul Stern) 63894 .RE 63895 .IP \[bu] 2 63896 build 63897 .RS 2 63898 .IP \[bu] 2 63899 Update to go1.22 and make go1.20 the minimum required version (Nick 63900 Craig-Wood) 63901 .IP \[bu] 2 63902 Fix \f[C]CVE-2024-24786\f[R] by upgrading 63903 \f[C]google.golang.org/protobuf\f[R] (Nick Craig-Wood) 63904 .RE 63905 .IP \[bu] 2 63906 check: Respect \f[C]--no-unicode-normalization\f[R] and 63907 \f[C]--ignore-case-sync\f[R] for \f[C]--checkfile\f[R] (nielash) 63908 .IP \[bu] 2 63909 cmd: Much improved shell auto completion which reduces the size of the 63910 completion file and works faster (Nick Craig-Wood) 63911 .IP \[bu] 2 63912 doc updates (albertony, ben-ba, Eli, emyarod, huajin tong, Jack 63913 Provance, kapitainsky, keongalvin, Nick Craig-Wood, nielash, rarspace01, 63914 rzitzer, Tera, Vincent Murphy) 63915 .IP \[bu] 2 63916 fs: Add more detailed logging for file includes/excludes (Kyle Reynolds) 63917 .IP \[bu] 2 63918 lsf 63919 .RS 2 63920 .IP \[bu] 2 63921 Add \f[C]--time-format\f[R] flag (nielash) 63922 .IP \[bu] 2 63923 Make metadata appear for directories (Nick Craig-Wood) 63924 .RE 63925 .IP \[bu] 2 63926 lsjson: Make metadata appear for directories (Nick Craig-Wood) 63927 .IP \[bu] 2 63928 rc 63929 .RS 2 63930 .IP \[bu] 2 63931 Add \f[C]srcFs\f[R] and \f[C]dstFs\f[R] to \f[C]core/stats\f[R] and 63932 \f[C]core/transferred\f[R] stats (Nick Craig-Wood) 63933 .IP \[bu] 2 63934 Add \f[C]operations/hashsum\f[R] to the rc as \f[C]rclone hashsum\f[R] 63935 equivalent (Nick Craig-Wood) 63936 .IP \[bu] 2 63937 Add \f[C]config/paths\f[R] to the rc as \f[C]rclone config paths\f[R] 63938 equivalent (Nick Craig-Wood) 63939 .RE 63940 .IP \[bu] 2 63941 sync 63942 .RS 2 63943 .IP \[bu] 2 63944 Optionally report list of synced paths to file (nielash) 63945 .IP \[bu] 2 63946 Implement directory sync for mod times and metadata (Nick Craig-Wood) 63947 .IP \[bu] 2 63948 Don\[aq]t set directory modtimes if already set (nielash) 63949 .IP \[bu] 2 63950 Don\[aq]t sync directory modtimes from backends which don\[aq]t have 63951 directories (Nick Craig-Wood) 63952 .RE 63953 .RE 63954 .IP \[bu] 2 63955 Bug Fixes 63956 .RS 2 63957 .IP \[bu] 2 63958 backend 63959 .RS 2 63960 .IP \[bu] 2 63961 Make backends which use oauth implement the \f[C]Shutdown\f[R] and 63962 shutdown the oauth properly (rkonfj) 63963 .RE 63964 .IP \[bu] 2 63965 bisync 63966 .RS 2 63967 .IP \[bu] 2 63968 Handle unicode and case normalization consistently (nielash) 63969 .IP \[bu] 2 63970 Partial uploads known issue on 63971 \f[C]local\f[R]/\f[C]ftp\f[R]/\f[C]sftp\f[R] has been resolved (unless 63972 using \f[C]--inplace\f[R]) (nielash) 63973 .IP \[bu] 2 63974 Fixed handling of unicode normalization and case insensitivity, support 63975 for \f[C]--fix-case\f[R] (https://rclone.org/docs/#fix-case), 63976 \f[C]--ignore-case-sync\f[R], \f[C]--no-unicode-normalization\f[R] 63977 (nielash) 63978 .IP \[bu] 2 63979 Bisync no longer fails to find the correct listing file when configs are 63980 overridden with backend-specific flags. 63981 (nielash) 63982 .RE 63983 .IP \[bu] 2 63984 nfsmount 63985 .RS 2 63986 .IP \[bu] 2 63987 Fix exit after external unmount (nielash) 63988 .IP \[bu] 2 63989 Fix \f[C]--volname\f[R] being ignored (nielash) 63990 .RE 63991 .IP \[bu] 2 63992 operations 63993 .RS 2 63994 .IP \[bu] 2 63995 Fix renaming a file on macOS (nielash) 63996 .IP \[bu] 2 63997 Fix case-insensitive moves in operations.Move (nielash) 63998 .IP \[bu] 2 63999 Fix TestCaseInsensitiveMoveFileDryRun on chunker integration tests 64000 (nielash) 64001 .IP \[bu] 2 64002 Fix TestMkdirModTime test (Nick Craig-Wood) 64003 .IP \[bu] 2 64004 Fix TestSetDirModTime for backends with SetDirModTime but not Metadata 64005 (Nick Craig-Wood) 64006 .IP \[bu] 2 64007 Fix typo in log messages (nielash) 64008 .RE 64009 .IP \[bu] 2 64010 serve nfs: Fix writing files via Finder on macOS (nielash) 64011 .IP \[bu] 2 64012 serve restic: Fix error handling (Michael Eischer) 64013 .IP \[bu] 2 64014 serve webdav: Fix \f[C]--baseurl\f[R] without leading / (Nick 64015 Craig-Wood) 64016 .IP \[bu] 2 64017 stats: Fix race between ResetCounters and stopAverageLoop called from 64018 time.AfterFunc (Nick Craig-Wood) 64019 .IP \[bu] 2 64020 sync 64021 .RS 2 64022 .IP \[bu] 2 64023 \f[C]--fix-case\f[R] flag to rename case insensitive dest (nielash) 64024 .IP \[bu] 2 64025 Use operations.DirMove instead of sync.MoveDir for \f[C]--fix-case\f[R] 64026 (nielash) 64027 .RE 64028 .IP \[bu] 2 64029 systemd: Fix detection and switch to the coreos package everywhere 64030 rather than having 2 separate libraries (Anagh Kumar Baranwal) 64031 .RE 64032 .IP \[bu] 2 64033 Mount 64034 .RS 2 64035 .IP \[bu] 2 64036 Fix macOS not noticing errors with \f[C]--daemon\f[R] (Nick Craig-Wood) 64037 .IP \[bu] 2 64038 Notice daemon dying much quicker (Nick Craig-Wood) 64039 .RE 64040 .IP \[bu] 2 64041 VFS 64042 .RS 2 64043 .IP \[bu] 2 64044 Fix unicode normalization on macOS (nielash) 64045 .RE 64046 .IP \[bu] 2 64047 Bisync 64048 .RS 2 64049 .IP \[bu] 2 64050 Copies and deletes are now handled in one operation instead of two 64051 (nielash) 64052 .IP \[bu] 2 64053 \f[C]--track-renames\f[R] and \f[C]--backup-dir\f[R] are now supported 64054 (nielash) 64055 .IP \[bu] 2 64056 Final listings are now generated from sync results, to avoid needing to 64057 re-list (nielash) 64058 .IP \[bu] 2 64059 Bisync is now much more resilient to changes that happen during a bisync 64060 run, and far less prone to critical errors / undetected changes 64061 (nielash) 64062 .IP \[bu] 2 64063 Bisync is now capable of rolling a file listing back in cases of 64064 uncertainty, essentially marking the file as needing to be rechecked 64065 next time. 64066 (nielash) 64067 .IP \[bu] 2 64068 A few basic terminal colors are now supported, controllable with 64069 \f[C]--color\f[R] (https://rclone.org/docs/#color-when) 64070 (\f[C]AUTO\f[R]|\f[C]NEVER\f[R]|\f[C]ALWAYS\f[R]) (nielash) 64071 .IP \[bu] 2 64072 Initial listing snapshots of Path1 and Path2 are now generated 64073 concurrently, using the same \[dq]march\[dq] infrastructure as 64074 \f[C]check\f[R] and \f[C]sync\f[R], for performance improvements and 64075 less risk of error. 64076 (nielash) 64077 .IP \[bu] 2 64078 \f[C]--resync\f[R] is now much more efficient (especially for users of 64079 \f[C]--create-empty-src-dirs\f[R]) (nielash) 64080 .IP \[bu] 2 64081 Google Docs (and other files of unknown size) are now supported (with 64082 the same options as in \f[C]sync\f[R]) (nielash) 64083 .IP \[bu] 2 64084 Equality checks before a sync conflict rename now fall back to 64085 \f[C]cryptcheck\f[R] (when possible) or \f[C]--download\f[R], (nielash) 64086 instead of of \f[C]--size-only\f[R], when \f[C]check\f[R] is not 64087 available. 64088 .IP \[bu] 2 64089 Bisync now fully supports comparing based on any combination of size, 64090 modtime, and checksum, lifting the prior restriction on backends without 64091 modtime support. 64092 (nielash) 64093 .IP \[bu] 2 64094 Bisync now supports a \[dq]Graceful Shutdown\[dq] mode to cleanly cancel 64095 a run early without requiring \f[C]--resync\f[R]. 64096 (nielash) 64097 .IP \[bu] 2 64098 New \f[C]--recover\f[R] flag allows robust recovery in the event of 64099 interruptions, without requiring \f[C]--resync\f[R]. 64100 (nielash) 64101 .IP \[bu] 2 64102 A new \f[C]--max-lock\f[R] setting allows lock files to automatically 64103 renew and expire, for better automatic recovery when a run is 64104 interrupted. 64105 (nielash) 64106 .IP \[bu] 2 64107 Bisync now supports auto-resolving sync conflicts and customizing rename 64108 behavior with new \f[C]--conflict-resolve\f[R], 64109 \f[C]--conflict-loser\f[R], and \f[C]--conflict-suffix\f[R] flags. 64110 (nielash) 64111 .IP \[bu] 2 64112 A new \f[C]--resync-mode\f[R] flag allows more control over which 64113 version of a file gets kept during a \f[C]--resync\f[R]. 64114 (nielash) 64115 .IP \[bu] 2 64116 Bisync now supports 64117 \f[C]--retries\f[R] (https://rclone.org/docs/#retries-int) and 64118 \f[C]--retries-sleep\f[R] (when \f[C]--resilient\f[R] is set.) (nielash) 64119 .IP \[bu] 2 64120 Clarify file operation directions in dry-run logs (Kyle Reynolds) 64121 .RE 64122 .IP \[bu] 2 64123 Local 64124 .RS 2 64125 .IP \[bu] 2 64126 Fix cleanRootPath on Windows after go1.21.4 stdlib update (nielash) 64127 .IP \[bu] 2 64128 Implement setting modification time on directories (nielash) 64129 .IP \[bu] 2 64130 Implement modtime and metadata for directories (Nick Craig-Wood) 64131 .IP \[bu] 2 64132 Fix setting of btime on directories on Windows (Nick Craig-Wood) 64133 .IP \[bu] 2 64134 Delete backend implementation of Purge to speed up and make stats (Nick 64135 Craig-Wood) 64136 .IP \[bu] 2 64137 Support metadata setting and mapping on server side Move (Nick 64138 Craig-Wood) 64139 .RE 64140 .IP \[bu] 2 64141 Cache 64142 .RS 2 64143 .IP \[bu] 2 64144 Implement setting modification time on directories (if supported by 64145 wrapped remote) (nielash) 64146 .IP \[bu] 2 64147 Implement setting metadata on directories (Nick Craig-Wood) 64148 .RE 64149 .IP \[bu] 2 64150 Crypt 64151 .RS 2 64152 .IP \[bu] 2 64153 Implement setting modification time on directories (if supported by 64154 wrapped remote) (nielash) 64155 .IP \[bu] 2 64156 Implement setting metadata on directories (Nick Craig-Wood) 64157 .IP \[bu] 2 64158 Improve handling of undecryptable file names (nielash) 64159 .IP \[bu] 2 64160 Add missing error check spotted by linter (Nick Craig-Wood) 64161 .RE 64162 .IP \[bu] 2 64163 Azure Blob 64164 .RS 2 64165 .IP \[bu] 2 64166 Implement \f[C]--azureblob-delete-snapshots\f[R] (Nick Craig-Wood) 64167 .RE 64168 .IP \[bu] 2 64169 B2 64170 .RS 2 64171 .IP \[bu] 2 64172 Clarify exactly what \f[C]--b2-download-auth-duration\f[R] does in the 64173 docs (Nick Craig-Wood) 64174 .RE 64175 .IP \[bu] 2 64176 Chunker 64177 .RS 2 64178 .IP \[bu] 2 64179 Implement setting modification time on directories (if supported by 64180 wrapped remote) (nielash) 64181 .IP \[bu] 2 64182 Implement setting metadata on directories (Nick Craig-Wood) 64183 .RE 64184 .IP \[bu] 2 64185 Combine 64186 .RS 2 64187 .IP \[bu] 2 64188 Implement setting modification time on directories (if supported by 64189 wrapped remote) (nielash) 64190 .IP \[bu] 2 64191 Implement setting metadata on directories (Nick Craig-Wood) 64192 .IP \[bu] 2 64193 Fix directory metadata error on upstream root (nielash) 64194 .IP \[bu] 2 64195 Fix directory move across upstreams (nielash) 64196 .RE 64197 .IP \[bu] 2 64198 Compress 64199 .RS 2 64200 .IP \[bu] 2 64201 Implement setting modification time on directories (if supported by 64202 wrapped remote) (nielash) 64203 .IP \[bu] 2 64204 Implement setting metadata on directories (Nick Craig-Wood) 64205 .RE 64206 .IP \[bu] 2 64207 Drive 64208 .RS 2 64209 .IP \[bu] 2 64210 Implement setting modification time on directories (nielash) 64211 .IP \[bu] 2 64212 Implement modtime and metadata setting for directories (Nick Craig-Wood) 64213 .IP \[bu] 2 64214 Support metadata setting and mapping on server side Move,Copy (Nick 64215 Craig-Wood) 64216 .RE 64217 .IP \[bu] 2 64218 FTP 64219 .RS 2 64220 .IP \[bu] 2 64221 Fix mkdir with rsftp which is returning the wrong code (Nick Craig-Wood) 64222 .RE 64223 .IP \[bu] 2 64224 Hasher 64225 .RS 2 64226 .IP \[bu] 2 64227 Implement setting modification time on directories (if supported by 64228 wrapped remote) (nielash) 64229 .IP \[bu] 2 64230 Implement setting metadata on directories (Nick Craig-Wood) 64231 .IP \[bu] 2 64232 Fix error from trying to stop an already-stopped db (nielash) 64233 .IP \[bu] 2 64234 Look for cached hash if passed hash unexpectedly blank (nielash) 64235 .RE 64236 .IP \[bu] 2 64237 Imagekit 64238 .RS 2 64239 .IP \[bu] 2 64240 Updated docs and web content (Harshit Budhraja) 64241 .IP \[bu] 2 64242 Updated overview - supported operations (Harshit Budhraja) 64243 .RE 64244 .IP \[bu] 2 64245 Mega 64246 .RS 2 64247 .IP \[bu] 2 64248 Fix panic with go1.22 (Nick Craig-Wood) 64249 .RE 64250 .IP \[bu] 2 64251 Netstorage 64252 .RS 2 64253 .IP \[bu] 2 64254 Fix Root to return correct directory when pointing to a file (Nick 64255 Craig-Wood) 64256 .RE 64257 .IP \[bu] 2 64258 Onedrive 64259 .RS 2 64260 .IP \[bu] 2 64261 Add metadata support (nielash) 64262 .RE 64263 .IP \[bu] 2 64264 Opendrive 64265 .RS 2 64266 .IP \[bu] 2 64267 Fix moving file/folder within the same parent dir (nielash) 64268 .RE 64269 .IP \[bu] 2 64270 Oracle Object Storage 64271 .RS 2 64272 .IP \[bu] 2 64273 Support \f[C]backend restore\f[R] command (Nikhil Ahuja) 64274 .IP \[bu] 2 64275 Support workload identity authentication for OKE (Anders Swanson) 64276 .RE 64277 .IP \[bu] 2 64278 Protondrive 64279 .RS 2 64280 .IP \[bu] 2 64281 Fix encoding of Root method (Nick Craig-Wood) 64282 .RE 64283 .IP \[bu] 2 64284 Quatrix 64285 .RS 2 64286 .IP \[bu] 2 64287 Fix \f[C]Content-Range\f[R] header (Volodymyr) 64288 .IP \[bu] 2 64289 Add option to skip project folders (Oksana Zhykina) 64290 .IP \[bu] 2 64291 Fix Root to return correct directory when pointing to a file (Nick 64292 Craig-Wood) 64293 .RE 64294 .IP \[bu] 2 64295 S3 64296 .RS 2 64297 .IP \[bu] 2 64298 Add \f[C]--s3-version-deleted\f[R] to show delete markers in listings 64299 when using versions. 64300 (Nick Craig-Wood) 64301 .IP \[bu] 2 64302 Add IPv6 support with option \f[C]--s3-use-dual-stack\f[R] (Anthony 64303 Metzidis) 64304 .IP \[bu] 2 64305 Copy parts in parallel when doing chunked server side copy (Nick 64306 Craig-Wood) 64307 .IP \[bu] 2 64308 GCS provider: fix server side copy of files bigger than 5G (Nick 64309 Craig-Wood) 64310 .IP \[bu] 2 64311 Support metadata setting and mapping on server side Copy (Nick 64312 Craig-Wood) 64313 .RE 64314 .IP \[bu] 2 64315 Seafile 64316 .RS 2 64317 .IP \[bu] 2 64318 Fix download/upload error when \f[C]FILE_SERVER_ROOT\f[R] is relative 64319 (DanielEgbers) 64320 .IP \[bu] 2 64321 Fix Root to return correct directory when pointing to a file (Nick 64322 Craig-Wood) 64323 .RE 64324 .IP \[bu] 2 64325 SFTP 64326 .RS 2 64327 .IP \[bu] 2 64328 Implement setting modification time on directories (nielash) 64329 .IP \[bu] 2 64330 Set directory modtimes update on write flag (Nick Craig-Wood) 64331 .IP \[bu] 2 64332 Shorten wait delay for external ssh binaries now that we are using 64333 go1.20 (Nick Craig-Wood) 64334 .RE 64335 .IP \[bu] 2 64336 Swift 64337 .RS 2 64338 .IP \[bu] 2 64339 Avoid unnecessary container versioning check (Joe Cai) 64340 .RE 64341 .IP \[bu] 2 64342 Union 64343 .RS 2 64344 .IP \[bu] 2 64345 Implement setting modification time on directories (if supported by 64346 wrapped remote) (nielash) 64347 .IP \[bu] 2 64348 Implement setting metadata on directories (Nick Craig-Wood) 64349 .RE 64350 .IP \[bu] 2 64351 WebDAV 64352 .RS 2 64353 .IP \[bu] 2 64354 Reduce priority of chunks upload log (Gabriel Ramos) 64355 .IP \[bu] 2 64356 owncloud: Add config \f[C]owncloud_exclude_shares\f[R] which allows to 64357 exclude shared files and folders when listing remote resources (Thomas 64358 M\[:u]ller) 64359 .RE 64360 .SS v1.65.2 - 2024-01-24 64361 .PP 64362 See commits (https://github.com/artpar/artpar/compare/v1.65.1...v1.65.2) 64363 .IP \[bu] 2 64364 Bug Fixes 64365 .RS 2 64366 .IP \[bu] 2 64367 build: bump github.com/cloudflare/circl from 1.3.6 to 1.3.7 (dependabot) 64368 .IP \[bu] 2 64369 docs updates (Nick Craig-Wood, kapitainsky, nielash, Tera, Harshit 64370 Budhraja) 64371 .RE 64372 .IP \[bu] 2 64373 VFS 64374 .RS 2 64375 .IP \[bu] 2 64376 Fix stale data when using \f[C]--vfs-cache-mode\f[R] full (Nick 64377 Craig-Wood) 64378 .RE 64379 .IP \[bu] 2 64380 Azure Blob 64381 .RS 2 64382 .IP \[bu] 2 64383 \f[B]IMPORTANT\f[R] Fix data corruption bug - see 64384 #7590 (https://github.com/artpar/artpar/issues/7590) (Nick Craig-Wood) 64385 .RE 64386 .SS v1.65.1 - 2024-01-08 64387 .PP 64388 See commits (https://github.com/artpar/artpar/compare/v1.65.0...v1.65.1) 64389 .IP \[bu] 2 64390 Bug Fixes 64391 .RS 2 64392 .IP \[bu] 2 64393 build 64394 .RS 2 64395 .IP \[bu] 2 64396 Bump golang.org/x/crypto to fix ssh terrapin CVE-2023-48795 (dependabot) 64397 .IP \[bu] 2 64398 Update to go1.21.5 to fix Windows path problems (Nick Craig-Wood) 64399 .IP \[bu] 2 64400 Fix docker build on arm/v6 (Nick Craig-Wood) 64401 .RE 64402 .IP \[bu] 2 64403 install.sh: fix harmless error message on install (Nick Craig-Wood) 64404 .IP \[bu] 2 64405 accounting: fix stats to show server side transfers (Nick Craig-Wood) 64406 .IP \[bu] 2 64407 doc fixes (albertony, ben-ba, Eli Orzitzer, emyarod, keongalvin, 64408 rarspace01) 64409 .IP \[bu] 2 64410 nfsmount: Compile for all unix oses, add \f[C]--sudo\f[R] and fix 64411 error/option handling (Nick Craig-Wood) 64412 .IP \[bu] 2 64413 operations: Fix files moved by rclone move not being counted as 64414 transfers (Nick Craig-Wood) 64415 .IP \[bu] 2 64416 oauthutil: Avoid panic when \f[C]*token\f[R] and \f[C]*ts.token\f[R] are 64417 the same (rkonfj) 64418 .IP \[bu] 2 64419 serve s3: Fix listing oddities (Nick Craig-Wood) 64420 .RE 64421 .IP \[bu] 2 64422 VFS 64423 .RS 2 64424 .IP \[bu] 2 64425 Note that \f[C]--vfs-refresh\f[R] runs in the background (Nick 64426 Craig-Wood) 64427 .RE 64428 .IP \[bu] 2 64429 Azurefiles 64430 .RS 2 64431 .IP \[bu] 2 64432 Fix storage base url (Oksana) 64433 .RE 64434 .IP \[bu] 2 64435 Crypt 64436 .RS 2 64437 .IP \[bu] 2 64438 Fix rclone move a file over itself deleting the file (Nick Craig-Wood) 64439 .RE 64440 .IP \[bu] 2 64441 Chunker 64442 .RS 2 64443 .IP \[bu] 2 64444 Fix rclone move a file over itself deleting the file (Nick Craig-Wood) 64445 .RE 64446 .IP \[bu] 2 64447 Compress 64448 .RS 2 64449 .IP \[bu] 2 64450 Fix rclone move a file over itself deleting the file (Nick Craig-Wood) 64451 .RE 64452 .IP \[bu] 2 64453 Dropbox 64454 .RS 2 64455 .IP \[bu] 2 64456 Fix used space on dropbox team accounts (Nick Craig-Wood) 64457 .RE 64458 .IP \[bu] 2 64459 FTP 64460 .RS 2 64461 .IP \[bu] 2 64462 Fix multi-thread copy (WeidiDeng) 64463 .RE 64464 .IP \[bu] 2 64465 Googlephotos 64466 .RS 2 64467 .IP \[bu] 2 64468 Fix nil pointer exception when batch failed (Nick Craig-Wood) 64469 .RE 64470 .IP \[bu] 2 64471 Hasher 64472 .RS 2 64473 .IP \[bu] 2 64474 Fix rclone move a file over itself deleting the file (Nick Craig-Wood) 64475 .IP \[bu] 2 64476 Fix invalid memory address error when MaxAge == 0 (nielash) 64477 .RE 64478 .IP \[bu] 2 64479 Onedrive 64480 .RS 2 64481 .IP \[bu] 2 64482 Fix error listing: unknown object type \f[C]<nil>\f[R] (Nick Craig-Wood) 64483 .IP \[bu] 2 64484 Fix \[dq]unauthenticated: Unauthenticated\[dq] errors when uploading 64485 (Nick Craig-Wood) 64486 .RE 64487 .IP \[bu] 2 64488 Oracleobjectstorage 64489 .RS 2 64490 .IP \[bu] 2 64491 Fix object storage endpoint for custom endpoints (Manoj Ghosh) 64492 .IP \[bu] 2 64493 Multipart copy create bucket if it doesn\[aq]t exist. 64494 (Manoj Ghosh) 64495 .RE 64496 .IP \[bu] 2 64497 Protondrive 64498 .RS 2 64499 .IP \[bu] 2 64500 Fix CVE-2023-45286 / GHSA-xwh9-gc39-5298 (Nick Craig-Wood) 64501 .RE 64502 .IP \[bu] 2 64503 S3 64504 .RS 2 64505 .IP \[bu] 2 64506 Fix crash if no UploadId in multipart upload (Nick Craig-Wood) 64507 .RE 64508 .IP \[bu] 2 64509 Smb 64510 .RS 2 64511 .IP \[bu] 2 64512 Fix shares not listed by updating go-smb2 (halms) 64513 .RE 64514 .IP \[bu] 2 64515 Union 64516 .RS 2 64517 .IP \[bu] 2 64518 Fix rclone move a file over itself deleting the file (Nick Craig-Wood) 64519 .RE 64520 .SS v1.65.0 - 2023-11-26 64521 .PP 64522 See commits (https://github.com/artpar/artpar/compare/v1.64.0...v1.65.0) 64523 .IP \[bu] 2 64524 New backends 64525 .RS 2 64526 .IP \[bu] 2 64527 Azure Files (karan, moongdal, Nick Craig-Wood) 64528 .IP \[bu] 2 64529 ImageKit (Abhinav Dhiman) 64530 .IP \[bu] 2 64531 Linkbox (viktor, Nick Craig-Wood) 64532 .RE 64533 .IP \[bu] 2 64534 New commands 64535 .RS 2 64536 .IP \[bu] 2 64537 \f[C]serve s3\f[R]: Let rclone act as an S3 compatible server (Mikubill, 64538 Artur Neumann, Saw-jan, Nick Craig-Wood) 64539 .IP \[bu] 2 64540 \f[C]nfsmount\f[R]: mount command to provide mount mechanism on macOS 64541 without FUSE (Saleh Dindar) 64542 .IP \[bu] 2 64543 \f[C]serve nfs\f[R]: to serve a remote for use by \f[C]nfsmount\f[R] 64544 (Saleh Dindar) 64545 .RE 64546 .IP \[bu] 2 64547 New Features 64548 .RS 2 64549 .IP \[bu] 2 64550 install.sh: Clean up temp files in install script (Jacob Hands) 64551 .IP \[bu] 2 64552 build 64553 .RS 2 64554 .IP \[bu] 2 64555 Update all dependencies (Nick Craig-Wood) 64556 .IP \[bu] 2 64557 Refactor version info and icon resource handling on windows (albertony) 64558 .RE 64559 .IP \[bu] 2 64560 doc updates (albertony, alfish2000, asdffdsazqqq, Dimitri Papadopoulos, 64561 Herby Gillot, Joda St\[:o]\[ss]er, Manoj Ghosh, Nick Craig-Wood) 64562 .IP \[bu] 2 64563 Implement \f[C]--metadata-mapper\f[R] to transform metatadata with a 64564 user supplied program (Nick Craig-Wood) 64565 .IP \[bu] 2 64566 Add \f[C]ChunkWriterDoesntSeek\f[R] feature flag and set it for b2 (Nick 64567 Craig-Wood) 64568 .IP \[bu] 2 64569 lib/http: Export basic go string functions for use in 64570 \f[C]--template\f[R] (Gabriel Espinoza) 64571 .IP \[bu] 2 64572 makefile: Use POSIX compatible install arguments (Mina Gali\['c]) 64573 .IP \[bu] 2 64574 operations 64575 .RS 2 64576 .IP \[bu] 2 64577 Use less memory when doing multithread uploads (Nick Craig-Wood) 64578 .IP \[bu] 2 64579 Implement \f[C]--partial-suffix\f[R] to control extension of temporary 64580 file names (Volodymyr) 64581 .RE 64582 .IP \[bu] 2 64583 rc 64584 .RS 2 64585 .IP \[bu] 2 64586 Add \f[C]operations/check\f[R] to the rc API (Nick Craig-Wood) 64587 .IP \[bu] 2 64588 Always report an error as JSON (Nick Craig-Wood) 64589 .IP \[bu] 2 64590 Set \f[C]Last-Modified\f[R] header for files served by 64591 \f[C]--rc-serve\f[R] (Nikita Shoshin) 64592 .RE 64593 .IP \[bu] 2 64594 size: Dont show duplicate object count when less than 1k (albertony) 64595 .RE 64596 .IP \[bu] 2 64597 Bug Fixes 64598 .RS 2 64599 .IP \[bu] 2 64600 fshttp: Fix \f[C]--contimeout\f[R] being ignored 64601 (\[u4F60]\[u77E5]\[u9053]\[u672A]\[u6765]\[u5417]) 64602 .IP \[bu] 2 64603 march: Fix excessive parallelism when using \f[C]--no-traverse\f[R] 64604 (Nick Craig-Wood) 64605 .IP \[bu] 2 64606 ncdu: Fix crash when re-entering changed directory after rescan (Nick 64607 Craig-Wood) 64608 .IP \[bu] 2 64609 operations 64610 .RS 2 64611 .IP \[bu] 2 64612 Fix overwrite of destination when multi-thread transfer fails (Nick 64613 Craig-Wood) 64614 .IP \[bu] 2 64615 Fix invalid UTF-8 when truncating file names when not using 64616 \f[C]--inplace\f[R] (Nick Craig-Wood) 64617 .RE 64618 .IP \[bu] 2 64619 serve dnla: Fix crash on graceful exit (wuxingzhong) 64620 .RE 64621 .IP \[bu] 2 64622 Mount 64623 .RS 2 64624 .IP \[bu] 2 64625 Disable mount for freebsd and alias cmount as mount on that platform 64626 (Nick Craig-Wood) 64627 .RE 64628 .IP \[bu] 2 64629 VFS 64630 .RS 2 64631 .IP \[bu] 2 64632 Add \f[C]--vfs-refresh\f[R] flag to read all the directories on start 64633 (Beyond Meat) 64634 .IP \[bu] 2 64635 Implement Name() method in WriteFileHandle and ReadFileHandle (Saleh 64636 Dindar) 64637 .IP \[bu] 2 64638 Add go-billy dependency and make sure vfs.Handle implements billy.File 64639 (Saleh Dindar) 64640 .IP \[bu] 2 64641 Error out early if can\[aq]t upload 0 length file (Nick Craig-Wood) 64642 .RE 64643 .IP \[bu] 2 64644 Local 64645 .RS 2 64646 .IP \[bu] 2 64647 Fix copying from Windows Volume Shadows (Nick Craig-Wood) 64648 .RE 64649 .IP \[bu] 2 64650 Azure Blob 64651 .RS 2 64652 .IP \[bu] 2 64653 Add support for cold tier (Ivan Yanitra) 64654 .RE 64655 .IP \[bu] 2 64656 B2 64657 .RS 2 64658 .IP \[bu] 2 64659 Implement \[dq]rclone backend lifecycle\[dq] to read and set bucket 64660 lifecycles (Nick Craig-Wood) 64661 .IP \[bu] 2 64662 Implement \f[C]--b2-lifecycle\f[R] to control lifecycle when creating 64663 buckets (Nick Craig-Wood) 64664 .IP \[bu] 2 64665 Fix listing all buckets when not needed (Nick Craig-Wood) 64666 .IP \[bu] 2 64667 Fix multi-thread upload with copyto going to wrong name (Nick 64668 Craig-Wood) 64669 .IP \[bu] 2 64670 Fix server side chunked copy when file size was exactly 64671 \f[C]--b2-copy-cutoff\f[R] (Nick Craig-Wood) 64672 .IP \[bu] 2 64673 Fix streaming chunked files an exact multiple of chunk size (Nick 64674 Craig-Wood) 64675 .RE 64676 .IP \[bu] 2 64677 Box 64678 .RS 2 64679 .IP \[bu] 2 64680 Filter more EventIDs when polling (David Sze) 64681 .IP \[bu] 2 64682 Add more logging for polling (David Sze) 64683 .IP \[bu] 2 64684 Fix performance problem reading metadata for single files (Nick 64685 Craig-Wood) 64686 .RE 64687 .IP \[bu] 2 64688 Drive 64689 .RS 2 64690 .IP \[bu] 2 64691 Add read/write metadata support (Nick Craig-Wood) 64692 .IP \[bu] 2 64693 Add support for SHA-1 and SHA-256 checksums (rinsuki) 64694 .IP \[bu] 2 64695 Add \f[C]--drive-show-all-gdocs\f[R] to allow unexportable gdocs to be 64696 server side copied (Nick Craig-Wood) 64697 .IP \[bu] 2 64698 Add a note that \f[C]--drive-scope\f[R] accepts comma-separated list of 64699 scopes (Keigo Imai) 64700 .IP \[bu] 2 64701 Fix error updating created time metadata on existing object (Nick 64702 Craig-Wood) 64703 .IP \[bu] 2 64704 Fix integration tests by enabling metadata support from the context 64705 (Nick Craig-Wood) 64706 .RE 64707 .IP \[bu] 2 64708 Dropbox 64709 .RS 2 64710 .IP \[bu] 2 64711 Factor batcher into lib/batcher (Nick Craig-Wood) 64712 .IP \[bu] 2 64713 Fix missing encoding for rclone purge (Nick Craig-Wood) 64714 .RE 64715 .IP \[bu] 2 64716 Google Cloud Storage 64717 .RS 2 64718 .IP \[bu] 2 64719 Fix 400 Bad request errors when using multi-thread copy (Nick 64720 Craig-Wood) 64721 .RE 64722 .IP \[bu] 2 64723 Googlephotos 64724 .RS 2 64725 .IP \[bu] 2 64726 Implement batcher for uploads (Nick Craig-Wood) 64727 .RE 64728 .IP \[bu] 2 64729 Hdfs 64730 .RS 2 64731 .IP \[bu] 2 64732 Added support for list of namenodes in hdfs remote config 64733 (Tayo-pasedaRJ) 64734 .RE 64735 .IP \[bu] 2 64736 HTTP 64737 .RS 2 64738 .IP \[bu] 2 64739 Implement set backend command to update running backend (Nick 64740 Craig-Wood) 64741 .IP \[bu] 2 64742 Enable methods used with WebDAV (Alen \[vS]iljak) 64743 .RE 64744 .IP \[bu] 2 64745 Jottacloud 64746 .RS 2 64747 .IP \[bu] 2 64748 Add support for reading and writing metadata (albertony) 64749 .RE 64750 .IP \[bu] 2 64751 Onedrive 64752 .RS 2 64753 .IP \[bu] 2 64754 Implement ListR method which gives \f[C]--fast-list\f[R] support (Nick 64755 Craig-Wood) 64756 .RS 2 64757 .IP \[bu] 2 64758 This must be enabled with the \f[C]--onedrive-delta\f[R] flag 64759 .RE 64760 .RE 64761 .IP \[bu] 2 64762 Quatrix 64763 .RS 2 64764 .IP \[bu] 2 64765 Add partial upload support (Oksana Zhykina) 64766 .IP \[bu] 2 64767 Overwrite files on conflict during server-side move (Oksana Zhykina) 64768 .RE 64769 .IP \[bu] 2 64770 S3 64771 .RS 2 64772 .IP \[bu] 2 64773 Add Linode provider (Nick Craig-Wood) 64774 .IP \[bu] 2 64775 Add docs on how to add a new provider (Nick Craig-Wood) 64776 .IP \[bu] 2 64777 Fix no error being returned when creating a bucket we don\[aq]t own 64778 (Nick Craig-Wood) 64779 .IP \[bu] 2 64780 Emit a debug message if anonymous credentials are in use (Nick 64781 Craig-Wood) 64782 .IP \[bu] 2 64783 Add \f[C]--s3-disable-multipart-uploads\f[R] flag (Nick Craig-Wood) 64784 .IP \[bu] 2 64785 Detect looping when using gcs and versions (Nick Craig-Wood) 64786 .RE 64787 .IP \[bu] 2 64788 SFTP 64789 .RS 2 64790 .IP \[bu] 2 64791 Implement \f[C]--sftp-copy-is-hardlink\f[R] to server side copy as 64792 hardlink (Nick Craig-Wood) 64793 .RE 64794 .IP \[bu] 2 64795 Smb 64796 .RS 2 64797 .IP \[bu] 2 64798 Fix incorrect \f[C]about\f[R] size by switching to 64799 \f[C]github.com/cloudsoda/go-smb2\f[R] fork (Nick Craig-Wood) 64800 .IP \[bu] 2 64801 Fix modtime of multithread uploads by setting PartialUploads (Nick 64802 Craig-Wood) 64803 .RE 64804 .IP \[bu] 2 64805 WebDAV 64806 .RS 2 64807 .IP \[bu] 2 64808 Added an rclone vendor to work with \f[C]rclone serve webdav\f[R] 64809 (Adithya Kumar) 64810 .RE 64811 .SS v1.64.2 - 2023-10-19 64812 .PP 64813 See commits (https://github.com/artpar/artpar/compare/v1.64.1...v1.64.2) 64814 .IP \[bu] 2 64815 Bug Fixes 64816 .RS 2 64817 .IP \[bu] 2 64818 selfupdate: Fix \[dq]invalid hashsum signature\[dq] error (Nick 64819 Craig-Wood) 64820 .IP \[bu] 2 64821 build: Fix docker build running out of space (Nick Craig-Wood) 64822 .RE 64823 .SS v1.64.1 - 2023-10-17 64824 .PP 64825 See commits (https://github.com/artpar/artpar/compare/v1.64.0...v1.64.1) 64826 .IP \[bu] 2 64827 Bug Fixes 64828 .RS 2 64829 .IP \[bu] 2 64830 cmd: Make \f[C]--progress\f[R] output logs in the same format as without 64831 (Nick Craig-Wood) 64832 .IP \[bu] 2 64833 docs fixes (Dimitri Papadopoulos Orfanos, Herby Gillot, Manoj Ghosh, 64834 Nick Craig-Wood) 64835 .IP \[bu] 2 64836 lsjson: Make sure we set the global metadata flag too (Nick Craig-Wood) 64837 .IP \[bu] 2 64838 operations 64839 .RS 2 64840 .IP \[bu] 2 64841 Ensure concurrency is no greater than the number of chunks (Pat 64842 Patterson) 64843 .IP \[bu] 2 64844 Fix OpenOptions ignored in copy if operation was a multiThreadCopy 64845 (Vitor Gomes) 64846 .IP \[bu] 2 64847 Fix error message on delete to have file name (Nick Craig-Wood) 64848 .RE 64849 .IP \[bu] 2 64850 serve sftp: Return not supported error for not supported commands (Nick 64851 Craig-Wood) 64852 .IP \[bu] 2 64853 build: Upgrade golang.org/x/net to v0.17.0 to fix HTTP/2 rapid reset 64854 (Nick Craig-Wood) 64855 .IP \[bu] 2 64856 pacer: Fix b2 deadlock by defaulting max connections to unlimited (Nick 64857 Craig-Wood) 64858 .RE 64859 .IP \[bu] 2 64860 Mount 64861 .RS 2 64862 .IP \[bu] 2 64863 Fix automount not detecting drive is ready (Nick Craig-Wood) 64864 .RE 64865 .IP \[bu] 2 64866 VFS 64867 .RS 2 64868 .IP \[bu] 2 64869 Fix update dir modification time (Saleh Dindar) 64870 .RE 64871 .IP \[bu] 2 64872 Azure Blob 64873 .RS 2 64874 .IP \[bu] 2 64875 Fix \[dq]fatal error: concurrent map writes\[dq] (Nick Craig-Wood) 64876 .RE 64877 .IP \[bu] 2 64878 B2 64879 .RS 2 64880 .IP \[bu] 2 64881 Fix multipart upload: corrupted on transfer: sizes differ XXX vs 0 (Nick 64882 Craig-Wood) 64883 .IP \[bu] 2 64884 Fix locking window when getting mutipart upload URL (Nick Craig-Wood) 64885 .IP \[bu] 2 64886 Fix server side copies greater than 4GB (Nick Craig-Wood) 64887 .IP \[bu] 2 64888 Fix chunked streaming uploads (Nick Craig-Wood) 64889 .IP \[bu] 2 64890 Reduce default \f[C]--b2-upload-concurrency\f[R] to 4 to reduce memory 64891 usage (Nick Craig-Wood) 64892 .RE 64893 .IP \[bu] 2 64894 Onedrive 64895 .RS 2 64896 .IP \[bu] 2 64897 Fix the configurator to allow \f[C]/teams/ID\f[R] in the config (Nick 64898 Craig-Wood) 64899 .RE 64900 .IP \[bu] 2 64901 Oracleobjectstorage 64902 .RS 2 64903 .IP \[bu] 2 64904 Fix OpenOptions being ignored in uploadMultipart with chunkWriter (Nick 64905 Craig-Wood) 64906 .RE 64907 .IP \[bu] 2 64908 S3 64909 .RS 2 64910 .IP \[bu] 2 64911 Fix slice bounds out of range error when listing (Nick Craig-Wood) 64912 .IP \[bu] 2 64913 Fix OpenOptions being ignored in uploadMultipart with chunkWriter (Vitor 64914 Gomes) 64915 .RE 64916 .IP \[bu] 2 64917 Storj 64918 .RS 2 64919 .IP \[bu] 2 64920 Update storj.io/uplink to v1.12.0 (Kaloyan Raev) 64921 .RE 64922 .SS v1.64.0 - 2023-09-11 64923 .PP 64924 See commits (https://github.com/artpar/artpar/compare/v1.63.0...v1.64.0) 64925 .IP \[bu] 2 64926 New backends 64927 .RS 2 64928 .IP \[bu] 2 64929 Proton Drive (https://rclone.org/protondrive/) (Chun-Hung Tseng) 64930 .IP \[bu] 2 64931 Quatrix (https://rclone.org/quatrix/) (Oksana, Volodymyr Kit) 64932 .IP \[bu] 2 64933 New S3 providers 64934 .RS 2 64935 .IP \[bu] 2 64936 Synology C2 (https://rclone.org/s3/#synology-c2) (BakaWang) 64937 .IP \[bu] 2 64938 Leviia (https://rclone.org/s3/#leviia) (Benjamin) 64939 .RE 64940 .IP \[bu] 2 64941 New Jottacloud providers 64942 .RS 2 64943 .IP \[bu] 2 64944 Onlime (https://rclone.org/jottacloud/) (Fjodor42) 64945 .IP \[bu] 2 64946 Telia Sky (https://rclone.org/jottacloud/) (NoLooseEnds) 64947 .RE 64948 .RE 64949 .IP \[bu] 2 64950 Major changes 64951 .RS 2 64952 .IP \[bu] 2 64953 Multi-thread transfers (Vitor Gomes, Nick Craig-Wood, Manoj Ghosh, Edwin 64954 Mackenzie-Owen) 64955 .RS 2 64956 .IP \[bu] 2 64957 Multi-thread transfers are now available when transferring to: 64958 .RS 2 64959 .IP \[bu] 2 64960 \f[C]local\f[R], \f[C]s3\f[R], \f[C]azureblob\f[R], \f[C]b2\f[R], 64961 \f[C]oracleobjectstorage\f[R] and \f[C]smb\f[R] 64962 .RE 64963 .IP \[bu] 2 64964 This greatly improves transfer speed between two network sources. 64965 .IP \[bu] 2 64966 In memory buffering has been unified between all backends and should 64967 share memory better. 64968 .IP \[bu] 2 64969 See --multi-thread docs (https://rclone.org/docs/#multi-thread-cutoff) 64970 for more info 64971 .RE 64972 .RE 64973 .IP \[bu] 2 64974 New commands 64975 .RS 2 64976 .IP \[bu] 2 64977 \f[C]rclone config redacted\f[R] support mechanism for showing redacted 64978 config (Nick Craig-Wood) 64979 .RE 64980 .IP \[bu] 2 64981 New Features 64982 .RS 2 64983 .IP \[bu] 2 64984 accounting 64985 .RS 2 64986 .IP \[bu] 2 64987 Show server side stats in own lines and not as bytes transferred (Nick 64988 Craig-Wood) 64989 .RE 64990 .IP \[bu] 2 64991 bisync 64992 .RS 2 64993 .IP \[bu] 2 64994 Add new \f[C]--ignore-listing-checksum\f[R] flag to distinguish from 64995 \f[C]--ignore-checksum\f[R] (nielash) 64996 .IP \[bu] 2 64997 Add experimental \f[C]--resilient\f[R] mode to allow recovery from 64998 self-correctable errors (nielash) 64999 .IP \[bu] 2 65000 Add support for \f[C]--create-empty-src-dirs\f[R] (nielash) 65001 .IP \[bu] 2 65002 Dry runs no longer commit filter changes (nielash) 65003 .IP \[bu] 2 65004 Enforce \f[C]--check-access\f[R] during \f[C]--resync\f[R] (nielash) 65005 .IP \[bu] 2 65006 Apply filters correctly during deletes (nielash) 65007 .IP \[bu] 2 65008 Equality check before renaming (leave identical files alone) (nielash) 65009 .IP \[bu] 2 65010 Fix \f[C]dryRun\f[R] rc parameter being ignored (nielash) 65011 .RE 65012 .IP \[bu] 2 65013 build 65014 .RS 2 65015 .IP \[bu] 2 65016 Update to \f[C]go1.21\f[R] and make \f[C]go1.19\f[R] the minimum 65017 required version (Anagh Kumar Baranwal, Nick Craig-Wood) 65018 .IP \[bu] 2 65019 Update dependencies (Nick Craig-Wood) 65020 .IP \[bu] 2 65021 Add snap installation (hideo aoyama) 65022 .IP \[bu] 2 65023 Change Winget Releaser job to \f[C]ubuntu-latest\f[R] (sitiom) 65024 .RE 65025 .IP \[bu] 2 65026 cmd: Refactor and use sysdnotify in more commands (eNV25) 65027 .IP \[bu] 2 65028 config: Add \f[C]--multi-thread-chunk-size\f[R] flag (Vitor Gomes) 65029 .IP \[bu] 2 65030 doc updates (antoinetran, Benjamin, Bj\[/o]rn Smith, Dean Attali, 65031 gabriel-suela, James Braza, Justin Hellings, kapitainsky, Mahad, 65032 Masamune3210, Nick Craig-Wood, Nihaal Sangha, Niklas Hamb\[:u]chen, 65033 Raymond Berger, r-ricci, Sawada Tsunayoshi, Tiago Boeing, Vladislav 65034 Vorobev) 65035 .IP \[bu] 2 65036 fs 65037 .RS 2 65038 .IP \[bu] 2 65039 Use atomic types everywhere (Roberto Ricci) 65040 .IP \[bu] 2 65041 When \f[C]--max-transfer\f[R] limit is reached exit with code (10) 65042 (kapitainsky) 65043 .IP \[bu] 2 65044 Add rclone completion powershell - basic implementation only (Nick 65045 Craig-Wood) 65046 .RE 65047 .IP \[bu] 2 65048 http servers: Allow CORS to be set with \f[C]--allow-origin\f[R] flag 65049 (yuudi) 65050 .IP \[bu] 2 65051 lib/rest: Remove unnecessary \f[C]nil\f[R] check (Eng Zer Jun) 65052 .IP \[bu] 2 65053 ncdu: Add keybinding to rescan filesystem (eNV25) 65054 .IP \[bu] 2 65055 rc 65056 .RS 2 65057 .IP \[bu] 2 65058 Add \f[C]executeId\f[R] to job listings (yuudi) 65059 .IP \[bu] 2 65060 Add \f[C]core/du\f[R] to measure local disk usage (Nick Craig-Wood) 65061 .IP \[bu] 2 65062 Add \f[C]operations/settier\f[R] to API (Drew Stinnett) 65063 .RE 65064 .IP \[bu] 2 65065 rclone test info: Add \f[C]--check-base32768\f[R] flag to check can 65066 store all base32768 characters (Nick Craig-Wood) 65067 .IP \[bu] 2 65068 rmdirs: Remove directories concurrently controlled by 65069 \f[C]--checkers\f[R] (Nick Craig-Wood) 65070 .RE 65071 .IP \[bu] 2 65072 Bug Fixes 65073 .RS 2 65074 .IP \[bu] 2 65075 accounting: Don\[aq]t stop calculating average transfer speed until the 65076 operation is complete (Jacob Hands) 65077 .IP \[bu] 2 65078 fs: Fix \f[C]transferTime\f[R] not being set in JSON logs (Jacob Hands) 65079 .IP \[bu] 2 65080 fshttp: Fix \f[C]--bind 0.0.0.0\f[R] allowing IPv6 and 65081 \f[C]--bind ::0\f[R] allowing IPv4 (Nick Craig-Wood) 65082 .IP \[bu] 2 65083 operations: Fix overlapping check on case insensitive file systems (Nick 65084 Craig-Wood) 65085 .IP \[bu] 2 65086 serve dlna: Fix MIME type if backend can\[aq]t identify it (Nick 65087 Craig-Wood) 65088 .IP \[bu] 2 65089 serve ftp: Fix race condition when using the auth proxy (Nick 65090 Craig-Wood) 65091 .IP \[bu] 2 65092 serve sftp: Fix hash calculations with \f[C]--vfs-cache-mode full\f[R] 65093 (Nick Craig-Wood) 65094 .IP \[bu] 2 65095 serve webdav: Fix error: Expecting fs.Object or fs.Directory, got 65096 \f[C]nil\f[R] (Nick Craig-Wood) 65097 .IP \[bu] 2 65098 sync: Fix lockup with \f[C]--cutoff-mode=soft\f[R] and 65099 \f[C]--max-duration\f[R] (Nick Craig-Wood) 65100 .RE 65101 .IP \[bu] 2 65102 Mount 65103 .RS 2 65104 .IP \[bu] 2 65105 fix: Mount parsing for linux (Anagh Kumar Baranwal) 65106 .RE 65107 .IP \[bu] 2 65108 VFS 65109 .RS 2 65110 .IP \[bu] 2 65111 Add \f[C]--vfs-cache-min-free-space\f[R] to control minimum free space 65112 on the disk containing the cache (Nick Craig-Wood) 65113 .IP \[bu] 2 65114 Added cache cleaner for directories to reduce memory usage (Anagh Kumar 65115 Baranwal) 65116 .IP \[bu] 2 65117 Update parent directory modtimes on vfs actions (David Pedersen) 65118 .IP \[bu] 2 65119 Keep virtual directory status accurate and reduce deadlock potential 65120 (Anagh Kumar Baranwal) 65121 .IP \[bu] 2 65122 Make sure struct field is aligned for atomic access (Roberto Ricci) 65123 .RE 65124 .IP \[bu] 2 65125 Local 65126 .RS 2 65127 .IP \[bu] 2 65128 Rmdir return an error if the path is not a dir (zjx20) 65129 .RE 65130 .IP \[bu] 2 65131 Azure Blob 65132 .RS 2 65133 .IP \[bu] 2 65134 Implement \f[C]OpenChunkWriter\f[R] and multi-thread uploads (Nick 65135 Craig-Wood) 65136 .IP \[bu] 2 65137 Fix creation of directory markers (Nick Craig-Wood) 65138 .IP \[bu] 2 65139 Fix purging with directory markers (Nick Craig-Wood) 65140 .RE 65141 .IP \[bu] 2 65142 B2 65143 .RS 2 65144 .IP \[bu] 2 65145 Implement \f[C]OpenChunkWriter\f[R] and multi-thread uploads (Nick 65146 Craig-Wood) 65147 .IP \[bu] 2 65148 Fix rclone link when object path contains special characters (Alishan 65149 Ladhani) 65150 .RE 65151 .IP \[bu] 2 65152 Box 65153 .RS 2 65154 .IP \[bu] 2 65155 Add polling support (David Sze) 65156 .IP \[bu] 2 65157 Add \f[C]--box-impersonate\f[R] to impersonate a user ID (Nick 65158 Craig-Wood) 65159 .IP \[bu] 2 65160 Fix unhelpful decoding of error messages into decimal numbers (Nick 65161 Craig-Wood) 65162 .RE 65163 .IP \[bu] 2 65164 Chunker 65165 .RS 2 65166 .IP \[bu] 2 65167 Update documentation to mention issue with small files (Ricardo D\[aq]O. 65168 Albanus) 65169 .RE 65170 .IP \[bu] 2 65171 Compress 65172 .RS 2 65173 .IP \[bu] 2 65174 Fix ChangeNotify (Nick Craig-Wood) 65175 .RE 65176 .IP \[bu] 2 65177 Drive 65178 .RS 2 65179 .IP \[bu] 2 65180 Add \f[C]--drive-fast-list-bug-fix\f[R] to control ListR bug workaround 65181 (Nick Craig-Wood) 65182 .RE 65183 .IP \[bu] 2 65184 Fichier 65185 .RS 2 65186 .IP \[bu] 2 65187 Implement \f[C]DirMove\f[R] (Nick Craig-Wood) 65188 .IP \[bu] 2 65189 Fix error code parsing (alexia) 65190 .RE 65191 .IP \[bu] 2 65192 FTP 65193 .RS 2 65194 .IP \[bu] 2 65195 Add socks_proxy support for SOCKS5 proxies (Zach) 65196 .IP \[bu] 2 65197 Fix 425 \[dq]TLS session of data connection not resumed\[dq] errors 65198 (Nick Craig-Wood) 65199 .RE 65200 .IP \[bu] 2 65201 Hdfs 65202 .RS 2 65203 .IP \[bu] 2 65204 Retry \[dq]replication in progress\[dq] errors when uploading (Nick 65205 Craig-Wood) 65206 .IP \[bu] 2 65207 Fix uploading to the wrong object on Update with overridden remote name 65208 (Nick Craig-Wood) 65209 .RE 65210 .IP \[bu] 2 65211 HTTP 65212 .RS 2 65213 .IP \[bu] 2 65214 CORS should not be sent if not set (yuudi) 65215 .IP \[bu] 2 65216 Fix webdav OPTIONS response (yuudi) 65217 .RE 65218 .IP \[bu] 2 65219 Opendrive 65220 .RS 2 65221 .IP \[bu] 2 65222 Fix List on a just deleted and remade directory (Nick Craig-Wood) 65223 .RE 65224 .IP \[bu] 2 65225 Oracleobjectstorage 65226 .RS 2 65227 .IP \[bu] 2 65228 Use rclone\[aq]s rate limiter in multipart transfers (Manoj Ghosh) 65229 .IP \[bu] 2 65230 Implement \f[C]OpenChunkWriter\f[R] and multi-thread uploads (Manoj 65231 Ghosh) 65232 .RE 65233 .IP \[bu] 2 65234 S3 65235 .RS 2 65236 .IP \[bu] 2 65237 Refactor multipart upload to use \f[C]OpenChunkWriter\f[R] and 65238 \f[C]ChunkWriter\f[R] (Vitor Gomes) 65239 .IP \[bu] 2 65240 Factor generic multipart upload into \f[C]lib/multipart\f[R] (Nick 65241 Craig-Wood) 65242 .IP \[bu] 2 65243 Fix purging of root directory with \f[C]--s3-directory-markers\f[R] 65244 (Nick Craig-Wood) 65245 .IP \[bu] 2 65246 Add \f[C]rclone backend set\f[R] command to update the running config 65247 (Nick Craig-Wood) 65248 .IP \[bu] 2 65249 Add \f[C]rclone backend restore-status\f[R] command (Nick Craig-Wood) 65250 .RE 65251 .IP \[bu] 2 65252 SFTP 65253 .RS 2 65254 .IP \[bu] 2 65255 Stop uploads re-using the same ssh connection to improve performance 65256 (Nick Craig-Wood) 65257 .IP \[bu] 2 65258 Add \f[C]--sftp-ssh\f[R] to specify an external ssh binary to use (Nick 65259 Craig-Wood) 65260 .IP \[bu] 2 65261 Add socks_proxy support for SOCKS5 proxies (Zach) 65262 .IP \[bu] 2 65263 Support dynamic \f[C]--sftp-path-override\f[R] (nielash) 65264 .IP \[bu] 2 65265 Fix spurious warning when using \f[C]--sftp-ssh\f[R] (Nick Craig-Wood) 65266 .RE 65267 .IP \[bu] 2 65268 Smb 65269 .RS 2 65270 .IP \[bu] 2 65271 Implement multi-threaded writes for copies to smb (Edwin Mackenzie-Owen) 65272 .RE 65273 .IP \[bu] 2 65274 Storj 65275 .RS 2 65276 .IP \[bu] 2 65277 Performance improvement for large file uploads (Kaloyan Raev) 65278 .RE 65279 .IP \[bu] 2 65280 Swift 65281 .RS 2 65282 .IP \[bu] 2 65283 Fix HEADing 0-length objects when \f[C]--swift-no-large-objects\f[R] set 65284 (Julian Lepinski) 65285 .RE 65286 .IP \[bu] 2 65287 Union 65288 .RS 2 65289 .IP \[bu] 2 65290 Add \f[C]:writback\f[R] to act as a simple cache (Nick Craig-Wood) 65291 .RE 65292 .IP \[bu] 2 65293 WebDAV 65294 .RS 2 65295 .IP \[bu] 2 65296 Nextcloud: fix segment violation in low-level retry (Paul) 65297 .RE 65298 .IP \[bu] 2 65299 Zoho 65300 .RS 2 65301 .IP \[bu] 2 65302 Remove Range requests workarounds to fix integration tests (Nick 65303 Craig-Wood) 65304 .RE 65305 .SS v1.63.1 - 2023-07-17 65306 .PP 65307 See commits (https://github.com/artpar/artpar/compare/v1.63.0...v1.63.1) 65308 .IP \[bu] 2 65309 Bug Fixes 65310 .RS 2 65311 .IP \[bu] 2 65312 build: Fix macos builds for versions < 12 (Anagh Kumar Baranwal) 65313 .IP \[bu] 2 65314 dirtree: Fix performance with large directories of directories and 65315 \f[C]--fast-list\f[R] (Nick Craig-Wood) 65316 .IP \[bu] 2 65317 operations 65318 .RS 2 65319 .IP \[bu] 2 65320 Fix deadlock when using \f[C]lsd\f[R]/\f[C]ls\f[R] with 65321 \f[C]--progress\f[R] (Nick Craig-Wood) 65322 .IP \[bu] 2 65323 Fix \f[C].rclonelink\f[R] files not being converted back to symlinks 65324 (Nick Craig-Wood) 65325 .RE 65326 .IP \[bu] 2 65327 doc fixes (Dean Attali, Mahad, Nick Craig-Wood, Sawada Tsunayoshi, 65328 Vladislav Vorobev) 65329 .RE 65330 .IP \[bu] 2 65331 Local 65332 .RS 2 65333 .IP \[bu] 2 65334 Fix partial directory read for corrupted filesystem (Nick Craig-Wood) 65335 .RE 65336 .IP \[bu] 2 65337 Box 65338 .RS 2 65339 .IP \[bu] 2 65340 Fix reconnect failing with HTTP 400 Bad Request (albertony) 65341 .RE 65342 .IP \[bu] 2 65343 Smb 65344 .RS 2 65345 .IP \[bu] 2 65346 Fix \[dq]Statfs failed: bucket or container name is needed\[dq] when 65347 mounting (Nick Craig-Wood) 65348 .RE 65349 .IP \[bu] 2 65350 WebDAV 65351 .RS 2 65352 .IP \[bu] 2 65353 Nextcloud: fix must use /dav/files/USER endpoint not /webdav error 65354 (Paul) 65355 .IP \[bu] 2 65356 Nextcloud chunking: add more guidance for the user to check the config 65357 (darix) 65358 .RE 65359 .SS v1.63.0 - 2023-06-30 65360 .PP 65361 See commits (https://github.com/artpar/artpar/compare/v1.62.0...v1.63.0) 65362 .IP \[bu] 2 65363 New backends 65364 .RS 2 65365 .IP \[bu] 2 65366 Pikpak (https://rclone.org/pikpak/) (wiserain) 65367 .IP \[bu] 2 65368 New S3 providers 65369 .RS 2 65370 .IP \[bu] 2 65371 petabox.io (https://rclone.org/s3/#petabox) (Andrei Smirnov) 65372 .IP \[bu] 2 65373 Google Cloud Storage (https://rclone.org/s3/#google-cloud-storage) 65374 (Anthony Pessy) 65375 .RE 65376 .IP \[bu] 2 65377 New WebDAV providers 65378 .RS 2 65379 .IP \[bu] 2 65380 Fastmail (https://rclone.org/webdav/#fastmail-files) (Arnavion) 65381 .RE 65382 .RE 65383 .IP \[bu] 2 65384 Major changes 65385 .RS 2 65386 .IP \[bu] 2 65387 Files will be copied to a temporary name ending in \f[C].partial\f[R] 65388 when copying to \f[C]local\f[R],\f[C]ftp\f[R],\f[C]sftp\f[R] then 65389 renamed at the end of the transfer. 65390 (Janne Hellsten, Nick Craig-Wood) 65391 .RS 2 65392 .IP \[bu] 2 65393 This helps with data integrity as we don\[aq]t delete the existing file 65394 until the new one is complete. 65395 .IP \[bu] 2 65396 It can be disabled with the --inplace (https://rclone.org/docs/#inplace) 65397 flag. 65398 .IP \[bu] 2 65399 This behaviour will also happen if the backend is wrapped, for example 65400 \f[C]sftp\f[R] wrapped with \f[C]crypt\f[R]. 65401 .RE 65402 .IP \[bu] 2 65403 The s3 (https://rclone.org/s3/#s3-directory-markers), azureblob and gcs 65404 backends now support directory markers so empty directories are 65405 supported (J\[u0101]nis Bebr\[u012B]tis, Nick Craig-Wood) 65406 .IP \[bu] 2 65407 The --default-time (https://rclone.org/docs/#default-time-time) flag now 65408 controls the unknown modification time of files/dirs (Nick Craig-Wood) 65409 .RS 2 65410 .IP \[bu] 2 65411 If a file or directory does not have a modification time rclone can read 65412 then rclone will display this fixed time instead. 65413 .IP \[bu] 2 65414 For the old behaviour use \f[C]--default-time 0s\f[R] which will set 65415 this time to the time rclone started up. 65416 .RE 65417 .RE 65418 .IP \[bu] 2 65419 New Features 65420 .RS 2 65421 .IP \[bu] 2 65422 build 65423 .RS 2 65424 .IP \[bu] 2 65425 Modernise linters in use and fixup all affected code (albertony) 65426 .IP \[bu] 2 65427 Push docker beta to GHCR (GitHub container registry) (Richard Tweed) 65428 .RE 65429 .IP \[bu] 2 65430 cat: Add \f[C]--separator\f[R] option to cat command (Loren Gordon) 65431 .IP \[bu] 2 65432 config 65433 .RS 2 65434 .IP \[bu] 2 65435 Do not remove/overwrite other files during config file save (albertony) 65436 .IP \[bu] 2 65437 Do not overwrite config file symbolic link (albertony) 65438 .IP \[bu] 2 65439 Stop \f[C]config create\f[R] making invalid config files (Nick 65440 Craig-Wood) 65441 .RE 65442 .IP \[bu] 2 65443 doc updates (Adam K, Aditya Basu, albertony, asdffdsazqqq, Damo, 65444 danielkrajnik, Dimitri Papadopoulos, dlitster, Drew Parsons, jumbi77, 65445 kapitainsky, mac-15, Mariusz Suchodolski, Nick Craig-Wood, NickIAm, 65446 Rintze Zelle, Stanislav Gromov, Tareq Sharafy, URenko, yuudi, Zach Kipp) 65447 .IP \[bu] 2 65448 fs 65449 .RS 2 65450 .IP \[bu] 2 65451 Add \f[C]size\f[R] to JSON logs when moving or copying an object (Nick 65452 Craig-Wood) 65453 .IP \[bu] 2 65454 Allow boolean features to be enabled with \f[C]--disable !Feature\f[R] 65455 (Nick Craig-Wood) 65456 .RE 65457 .IP \[bu] 2 65458 genautocomplete: Rename to \f[C]completion\f[R] with alias to the old 65459 name (Nick Craig-Wood) 65460 .IP \[bu] 2 65461 librclone: Added example on using \f[C]librclone\f[R] with Go (alankrit) 65462 .IP \[bu] 2 65463 lsjson: Make \f[C]--stat\f[R] more efficient (Nick Craig-Wood) 65464 .IP \[bu] 2 65465 operations 65466 .RS 2 65467 .IP \[bu] 2 65468 Implement \f[C]--multi-thread-write-buffer-size\f[R] for speed 65469 improvements on downloads (Paulo Schreiner) 65470 .IP \[bu] 2 65471 Reopen downloads on error when using \f[C]check --download\f[R] and 65472 \f[C]cat\f[R] (Nick Craig-Wood) 65473 .RE 65474 .IP \[bu] 2 65475 rc: \f[C]config/listremotes\f[R] includes remotes defined with 65476 environment variables (kapitainsky) 65477 .IP \[bu] 2 65478 selfupdate: Obey \f[C]--no-check-certificate\f[R] flag (Nick Craig-Wood) 65479 .IP \[bu] 2 65480 serve restic: Trigger systemd notify (Shyim) 65481 .IP \[bu] 2 65482 serve webdav: Implement owncloud checksum and modtime extensions 65483 (WeidiDeng) 65484 .IP \[bu] 2 65485 sync: \f[C]--suffix-keep-extension\f[R] preserve 2 part extensions like 65486 \&.tar.gz (Nick Craig-Wood) 65487 .RE 65488 .IP \[bu] 2 65489 Bug Fixes 65490 .RS 2 65491 .IP \[bu] 2 65492 accounting 65493 .RS 2 65494 .IP \[bu] 2 65495 Fix Prometheus metrics to be the same as \f[C]core/stats\f[R] (Nick 65496 Craig-Wood) 65497 .IP \[bu] 2 65498 Bwlimit signal handler should always start (Sam Lai) 65499 .RE 65500 .IP \[bu] 2 65501 bisync: Fix \f[C]maxDelete\f[R] parameter being ignored via the rc (Nick 65502 Craig-Wood) 65503 .IP \[bu] 2 65504 cmd/ncdu: Fix screen corruption when logging (eNV25) 65505 .IP \[bu] 2 65506 filter: Fix deadlock with errors on \f[C]--files-from\f[R] (douchen) 65507 .IP \[bu] 2 65508 fs 65509 .RS 2 65510 .IP \[bu] 2 65511 Fix interaction between \f[C]--progress\f[R] and \f[C]--interactive\f[R] 65512 (Nick Craig-Wood) 65513 .IP \[bu] 2 65514 Fix infinite recursive call in pacer ModifyCalculator (fixes issue 65515 reported by the staticcheck linter) (albertony) 65516 .RE 65517 .IP \[bu] 2 65518 lib/atexit: Ensure OnError only calls cancel function once (Nick 65519 Craig-Wood) 65520 .IP \[bu] 2 65521 lib/rest: Fix problems re-using HTTP connections (Nick Craig-Wood) 65522 .IP \[bu] 2 65523 rc 65524 .RS 2 65525 .IP \[bu] 2 65526 Fix \f[C]operations/stat\f[R] with trailing \f[C]/\f[R] (Nick 65527 Craig-Wood) 65528 .IP \[bu] 2 65529 Fix missing \f[C]--rc\f[R] flags (Nick Craig-Wood) 65530 .IP \[bu] 2 65531 Fix output of Time values in \f[C]options/get\f[R] (Nick Craig-Wood) 65532 .RE 65533 .IP \[bu] 2 65534 serve dlna: Fix potential data race (Nick Craig-Wood) 65535 .IP \[bu] 2 65536 version: Fix reported os/kernel version for windows (albertony) 65537 .RE 65538 .IP \[bu] 2 65539 Mount 65540 .RS 2 65541 .IP \[bu] 2 65542 Add \f[C]--mount-case-insensitive\f[R] to force the mount to be case 65543 insensitive (Nick Craig-Wood) 65544 .IP \[bu] 2 65545 Removed unnecessary byte slice allocation for reads (Anagh Kumar 65546 Baranwal) 65547 .IP \[bu] 2 65548 Clarify rclone mount error when installed via homebrew (Nick Craig-Wood) 65549 .IP \[bu] 2 65550 Added _netdev to the example mount so it gets treated as a remote-fs 65551 rather than local-fs (Anagh Kumar Baranwal) 65552 .RE 65553 .IP \[bu] 2 65554 Mount2 65555 .RS 2 65556 .IP \[bu] 2 65557 Updated go-fuse version (Anagh Kumar Baranwal) 65558 .IP \[bu] 2 65559 Fixed statfs (Anagh Kumar Baranwal) 65560 .IP \[bu] 2 65561 Disable xattrs (Anagh Kumar Baranwal) 65562 .RE 65563 .IP \[bu] 2 65564 VFS 65565 .RS 2 65566 .IP \[bu] 2 65567 Add MkdirAll function to make a directory and all beneath (Nick 65568 Craig-Wood) 65569 .IP \[bu] 2 65570 Fix reload: failed to add virtual dir entry: file does not exist (Nick 65571 Craig-Wood) 65572 .IP \[bu] 2 65573 Fix writing to a read only directory creating spurious directory entries 65574 (WeidiDeng) 65575 .IP \[bu] 2 65576 Fix potential data race (Nick Craig-Wood) 65577 .IP \[bu] 2 65578 Fix backends being Shutdown too early when startup takes a long time 65579 (Nick Craig-Wood) 65580 .RE 65581 .IP \[bu] 2 65582 Local 65583 .RS 2 65584 .IP \[bu] 2 65585 Fix filtering of symlinks with \f[C]-l\f[R]/\f[C]--links\f[R] flag (Nick 65586 Craig-Wood) 65587 .IP \[bu] 2 65588 Fix /path/to/file.rclonelink when \f[C]-l\f[R]/\f[C]--links\f[R] is in 65589 use (Nick Craig-Wood) 65590 .IP \[bu] 2 65591 Fix crash with \f[C]--metadata\f[R] on Android (Nick Craig-Wood) 65592 .RE 65593 .IP \[bu] 2 65594 Cache 65595 .RS 2 65596 .IP \[bu] 2 65597 Fix backends shutting down when in use when used via the rc (Nick 65598 Craig-Wood) 65599 .RE 65600 .IP \[bu] 2 65601 Crypt 65602 .RS 2 65603 .IP \[bu] 2 65604 Add \f[C]--crypt-suffix\f[R] option to set a custom suffix for encrypted 65605 files (jladbrook) 65606 .IP \[bu] 2 65607 Add \f[C]--crypt-pass-bad-blocks\f[R] to allow corrupted file output 65608 (Nick Craig-Wood) 65609 .IP \[bu] 2 65610 Fix reading 0 length files (Nick Craig-Wood) 65611 .IP \[bu] 2 65612 Try not to return \[dq]unexpected EOF\[dq] error (Nick Craig-Wood) 65613 .IP \[bu] 2 65614 Reduce allocations (albertony) 65615 .IP \[bu] 2 65616 Recommend Dropbox for \f[C]base32768\f[R] encoding (Nick Craig-Wood) 65617 .RE 65618 .IP \[bu] 2 65619 Azure Blob 65620 .RS 2 65621 .IP \[bu] 2 65622 Empty directory markers (Nick Craig-Wood) 65623 .IP \[bu] 2 65624 Support azure workload identities (Tareq Sharafy) 65625 .IP \[bu] 2 65626 Fix azure blob uploads with multiple bits of metadata (Nick Craig-Wood) 65627 .IP \[bu] 2 65628 Fix azurite compatibility by sending nil tier if set to empty string 65629 (Roel Arents) 65630 .RE 65631 .IP \[bu] 2 65632 Combine 65633 .RS 2 65634 .IP \[bu] 2 65635 Implement missing methods (Nick Craig-Wood) 65636 .IP \[bu] 2 65637 Fix goroutine stack overflow on bad object (Nick Craig-Wood) 65638 .RE 65639 .IP \[bu] 2 65640 Drive 65641 .RS 2 65642 .IP \[bu] 2 65643 Add \f[C]--drive-env-auth\f[R] to get IAM credentials from runtime 65644 (Peter Brunner) 65645 .IP \[bu] 2 65646 Update drive service account guide (Juang, Yi-Lin) 65647 .IP \[bu] 2 65648 Fix change notify picking up files outside the root (Nick Craig-Wood) 65649 .IP \[bu] 2 65650 Fix trailing slash mis-identificaton of folder as file (Nick Craig-Wood) 65651 .IP \[bu] 2 65652 Fix incorrect remote after Update on object (Nick Craig-Wood) 65653 .RE 65654 .IP \[bu] 2 65655 Dropbox 65656 .RS 2 65657 .IP \[bu] 2 65658 Implement \f[C]--dropbox-pacer-min-sleep\f[R] flag (Nick Craig-Wood) 65659 .IP \[bu] 2 65660 Fix the dropbox batcher stalling (Misty) 65661 .RE 65662 .IP \[bu] 2 65663 Fichier 65664 .RS 2 65665 .IP \[bu] 2 65666 Add \f[C]--ficicher-cdn\f[R] option to use the CDN for download (Nick 65667 Craig-Wood) 65668 .RE 65669 .IP \[bu] 2 65670 FTP 65671 .RS 2 65672 .IP \[bu] 2 65673 Lower log message priority when \f[C]SetModTime\f[R] is not supported to 65674 debug (Tobias Gion) 65675 .IP \[bu] 2 65676 Fix \[dq]unsupported LIST line\[dq] errors on startup (Nick Craig-Wood) 65677 .IP \[bu] 2 65678 Fix \[dq]501 Not a valid pathname.\[dq] errors when creating directories 65679 (Nick Craig-Wood) 65680 .RE 65681 .IP \[bu] 2 65682 Google Cloud Storage 65683 .RS 2 65684 .IP \[bu] 2 65685 Empty directory markers (J\[u0101]nis Bebr\[u012B]tis, Nick Craig-Wood) 65686 .IP \[bu] 2 65687 Added \f[C]--gcs-user-project\f[R] needed for requester pays 65688 (Christopher Merry) 65689 .RE 65690 .IP \[bu] 2 65691 HTTP 65692 .RS 2 65693 .IP \[bu] 2 65694 Add client certificate user auth middleware. 65695 This can auth \f[C]serve restic\f[R] from the username in the client 65696 cert. 65697 (Peter Fern) 65698 .RE 65699 .IP \[bu] 2 65700 Jottacloud 65701 .RS 2 65702 .IP \[bu] 2 65703 Fix vfs writeback stuck in a failed upload loop with file versioning 65704 disabled (albertony) 65705 .RE 65706 .IP \[bu] 2 65707 Onedrive 65708 .RS 2 65709 .IP \[bu] 2 65710 Add \f[C]--onedrive-av-override\f[R] flag to download files flagged as 65711 virus (Nick Craig-Wood) 65712 .IP \[bu] 2 65713 Fix quickxorhash on 32 bit architectures (Nick Craig-Wood) 65714 .IP \[bu] 2 65715 Report any list errors during \f[C]rclone cleanup\f[R] (albertony) 65716 .RE 65717 .IP \[bu] 2 65718 Putio 65719 .RS 2 65720 .IP \[bu] 2 65721 Fix uploading to the wrong object on Update with overridden remote name 65722 (Nick Craig-Wood) 65723 .IP \[bu] 2 65724 Fix modification times not being preserved for server side copy and move 65725 (Nick Craig-Wood) 65726 .IP \[bu] 2 65727 Fix server side copy failures (400 errors) (Nick Craig-Wood) 65728 .RE 65729 .IP \[bu] 2 65730 S3 65731 .RS 2 65732 .IP \[bu] 2 65733 Empty directory markers (J\[u0101]nis Bebr\[u012B]tis, Nick Craig-Wood) 65734 .IP \[bu] 2 65735 Update Scaleway storage classes (Brian Starkey) 65736 .IP \[bu] 2 65737 Fix \f[C]--s3-versions\f[R] on individual objects (Nick Craig-Wood) 65738 .IP \[bu] 2 65739 Fix hang on aborting multipart upload with iDrive e2 (Nick Craig-Wood) 65740 .IP \[bu] 2 65741 Fix missing \[dq]tier\[dq] metadata (Nick Craig-Wood) 65742 .IP \[bu] 2 65743 Fix V3sign: add missing subresource delete (cc) 65744 .IP \[bu] 2 65745 Fix Arvancloud Domain and region changes and alphabetise the provider 65746 (Ehsan Tadayon) 65747 .IP \[bu] 2 65748 Fix Qiniu KODO quirks virtualHostStyle is false (zzq) 65749 .RE 65750 .IP \[bu] 2 65751 SFTP 65752 .RS 2 65753 .IP \[bu] 2 65754 Add \f[C]--sftp-host-key-algorithms\f[R] to allow specifying SSH host 65755 key algorithms (Joel) 65756 .IP \[bu] 2 65757 Fix using \f[C]--sftp-key-use-agent\f[R] and \f[C]--sftp-key-file\f[R] 65758 together needing private key file (Arnav Singh) 65759 .IP \[bu] 2 65760 Fix move to allow overwriting existing files (Nick Craig-Wood) 65761 .IP \[bu] 2 65762 Don\[aq]t stat directories before listing them (Nick Craig-Wood) 65763 .IP \[bu] 2 65764 Don\[aq]t check remote points to a file if it ends with / (Nick 65765 Craig-Wood) 65766 .RE 65767 .IP \[bu] 2 65768 Sharefile 65769 .RS 2 65770 .IP \[bu] 2 65771 Disable streamed transfers as they no longer work (Nick Craig-Wood) 65772 .RE 65773 .IP \[bu] 2 65774 Smb 65775 .RS 2 65776 .IP \[bu] 2 65777 Code cleanup to avoid overwriting ctx before first use (fixes issue 65778 reported by the staticcheck linter) (albertony) 65779 .RE 65780 .IP \[bu] 2 65781 Storj 65782 .RS 2 65783 .IP \[bu] 2 65784 Fix \[dq]uplink: too many requests\[dq] errors when uploading to the 65785 same file (Nick Craig-Wood) 65786 .IP \[bu] 2 65787 Fix uploading to the wrong object on Update with overridden remote name 65788 (Nick Craig-Wood) 65789 .RE 65790 .IP \[bu] 2 65791 Swift 65792 .RS 2 65793 .IP \[bu] 2 65794 Ignore 404 error when deleting an object (Nick Craig-Wood) 65795 .RE 65796 .IP \[bu] 2 65797 Union 65798 .RS 2 65799 .IP \[bu] 2 65800 Implement missing methods (Nick Craig-Wood) 65801 .IP \[bu] 2 65802 Allow errors to be unwrapped for inspection (Nick Craig-Wood) 65803 .RE 65804 .IP \[bu] 2 65805 Uptobox 65806 .RS 2 65807 .IP \[bu] 2 65808 Add \f[C]--uptobox-private\f[R] flag to make all uploaded files private 65809 (Nick Craig-Wood) 65810 .IP \[bu] 2 65811 Fix improper regex (Aaron Gokaslan) 65812 .IP \[bu] 2 65813 Fix Update returning the wrong object (Nick Craig-Wood) 65814 .IP \[bu] 2 65815 Fix rmdir declaring that directories weren\[aq]t empty (Nick Craig-Wood) 65816 .RE 65817 .IP \[bu] 2 65818 WebDAV 65819 .RS 2 65820 .IP \[bu] 2 65821 nextcloud: Add support for chunked uploads (Paul) 65822 .IP \[bu] 2 65823 Set modtime using propset for owncloud and nextcloud (WeidiDeng) 65824 .IP \[bu] 2 65825 Make pacer minSleep configurable with \f[C]--webdav-pacer-min-sleep\f[R] 65826 (ed) 65827 .IP \[bu] 2 65828 Fix server side copy/move not overwriting (WeidiDeng) 65829 .IP \[bu] 2 65830 Fix modtime on server side copy for owncloud and nextcloud (Nick 65831 Craig-Wood) 65832 .RE 65833 .IP \[bu] 2 65834 Yandex 65835 .RS 2 65836 .IP \[bu] 2 65837 Fix 400 Bad Request on transfer failure (Nick Craig-Wood) 65838 .RE 65839 .IP \[bu] 2 65840 Zoho 65841 .RS 2 65842 .IP \[bu] 2 65843 Fix downloads with \f[C]Range:\f[R] header returning the wrong data 65844 (Nick Craig-Wood) 65845 .RE 65846 .SS v1.62.2 - 2023-03-16 65847 .PP 65848 See commits (https://github.com/artpar/artpar/compare/v1.62.1...v1.62.2) 65849 .IP \[bu] 2 65850 Bug Fixes 65851 .RS 2 65852 .IP \[bu] 2 65853 docker volume plugin: Add missing fuse3 dependency (Nick Craig-Wood) 65854 .IP \[bu] 2 65855 docs: Fix size documentation (asdffdsazqqq) 65856 .RE 65857 .IP \[bu] 2 65858 FTP 65859 .RS 2 65860 .IP \[bu] 2 65861 Fix 426 errors on downloads with vsftpd (Lesmiscore) 65862 .RE 65863 .SS v1.62.1 - 2023-03-15 65864 .PP 65865 See commits (https://github.com/artpar/artpar/compare/v1.62.0...v1.62.1) 65866 .IP \[bu] 2 65867 Bug Fixes 65868 .RS 2 65869 .IP \[bu] 2 65870 docker: Add missing fuse3 dependency (cycneuramus) 65871 .IP \[bu] 2 65872 build: Update release docs to be more careful with the tag (Nick 65873 Craig-Wood) 65874 .IP \[bu] 2 65875 build: Set Github release to draft while uploading binaries (Nick 65876 Craig-Wood) 65877 .RE 65878 .SS v1.62.0 - 2023-03-14 65879 .PP 65880 See commits (https://github.com/artpar/artpar/compare/v1.61.0...v1.62.0) 65881 .IP \[bu] 2 65882 New Features 65883 .RS 2 65884 .IP \[bu] 2 65885 accounting: Make checkers show what they are doing (Nick Craig-Wood) 65886 .IP \[bu] 2 65887 authorize: Add support for custom templates (Hunter Wittenborn) 65888 .IP \[bu] 2 65889 build 65890 .RS 2 65891 .IP \[bu] 2 65892 Update to go1.20 (Nick Craig-Wood, Anagh Kumar Baranwal) 65893 .IP \[bu] 2 65894 Add winget releaser workflow (Ryan Caezar Itang) 65895 .IP \[bu] 2 65896 Add dependabot (Ryan Caezar Itang) 65897 .RE 65898 .IP \[bu] 2 65899 doc updates (albertony, Bryan Kaplan, Gerard Bosch, IMTheNachoMan, 65900 Justin Winokur, Manoj Ghosh, Nick Craig-Wood, Ole Frost, Peter Brunner, 65901 piyushgarg, Ryan Caezar Itang, Simmon Li, ToBeFree) 65902 .IP \[bu] 2 65903 filter: Emit INFO message when can\[aq]t work out directory filters 65904 (Nick Craig-Wood) 65905 .IP \[bu] 2 65906 fs 65907 .RS 2 65908 .IP \[bu] 2 65909 Added multiple ca certificate support. 65910 (alankrit) 65911 .IP \[bu] 2 65912 Add \f[C]--max-delete-size\f[R] a delete size threshold (Leandro 65913 Sacchet) 65914 .RE 65915 .IP \[bu] 2 65916 fspath: Allow the symbols \f[C]\[at]\f[R] and \f[C]+\f[R] in remote 65917 names (albertony) 65918 .IP \[bu] 2 65919 lib/terminal: Enable windows console virtual terminal sequences 65920 processing (ANSI/VT100 colors) (albertony) 65921 .IP \[bu] 2 65922 move: If \f[C]--check-first\f[R] and \f[C]--order-by\f[R] are set then 65923 delete with perfect ordering (Nick Craig-Wood) 65924 .IP \[bu] 2 65925 serve http: Support \f[C]--auth-proxy\f[R] (Matthias Baur) 65926 .RE 65927 .IP \[bu] 2 65928 Bug Fixes 65929 .RS 2 65930 .IP \[bu] 2 65931 accounting 65932 .RS 2 65933 .IP \[bu] 2 65934 Avoid negative ETA values for very slow speeds (albertony) 65935 .IP \[bu] 2 65936 Limit length of ETA string (albertony) 65937 .IP \[bu] 2 65938 Show human readable elapsed time when longer than a day (albertony) 65939 .RE 65940 .IP \[bu] 2 65941 all: Apply codeql fixes (Aaron Gokaslan) 65942 .IP \[bu] 2 65943 build 65944 .RS 2 65945 .IP \[bu] 2 65946 Fix condition for manual workflow run (albertony) 65947 .IP \[bu] 2 65948 Fix building for ARMv5 and ARMv6 (albertony) 65949 .RS 2 65950 .IP \[bu] 2 65951 selfupdate: Consider ARM version 65952 .IP \[bu] 2 65953 install.sh: fix ARMv6 download 65954 .IP \[bu] 2 65955 version: Report ARM version 65956 .RE 65957 .RE 65958 .IP \[bu] 2 65959 deletefile: Return error code 4 if file does not exist (Nick Craig-Wood) 65960 .IP \[bu] 2 65961 docker: Fix volume plugin does not remount volume on docker restart 65962 (logopk) 65963 .IP \[bu] 2 65964 fs: Fix race conditions in \f[C]--max-delete\f[R] and 65965 \f[C]--max-delete-size\f[R] (Nick Craig-Wood) 65966 .IP \[bu] 2 65967 lib/oauthutil: Handle fatal errors better (Alex Chen) 65968 .IP \[bu] 2 65969 mount2: Fix \f[C]--allow-non-empty\f[R] (Nick Craig-Wood) 65970 .IP \[bu] 2 65971 operations: Fix concurrency: use \f[C]--checkers\f[R] unless 65972 transferring files (Nick Craig-Wood) 65973 .IP \[bu] 2 65974 serve ftp: Fix timestamps older than 1 year in listings (Nick 65975 Craig-Wood) 65976 .IP \[bu] 2 65977 sync: Fix concurrency: use \f[C]--checkers\f[R] unless transferring 65978 files (Nick Craig-Wood) 65979 .IP \[bu] 2 65980 tree 65981 .RS 2 65982 .IP \[bu] 2 65983 Fix nil pointer exception on stat failure (Nick Craig-Wood) 65984 .IP \[bu] 2 65985 Fix colored output on windows (albertony) 65986 .IP \[bu] 2 65987 Fix display of files with illegal Windows file system names (Nick 65988 Craig-Wood) 65989 .RE 65990 .RE 65991 .IP \[bu] 2 65992 Mount 65993 .RS 2 65994 .IP \[bu] 2 65995 Fix creating and renaming files on case insensitive backends (Nick 65996 Craig-Wood) 65997 .IP \[bu] 2 65998 Do not treat \f[C]\[rs]\[rs]?\[rs]\f[R] prefixed paths as network share 65999 paths on windows (albertony) 66000 .IP \[bu] 2 66001 Fix check for empty mount point on Linux (Nick Craig-Wood) 66002 .IP \[bu] 2 66003 Fix \f[C]--allow-non-empty\f[R] (Nick Craig-Wood) 66004 .IP \[bu] 2 66005 Avoid incorrect or premature overlap check on windows (albertony) 66006 .IP \[bu] 2 66007 Update to fuse3 after bazil.org/fuse update (Nick Craig-Wood) 66008 .RE 66009 .IP \[bu] 2 66010 VFS 66011 .RS 2 66012 .IP \[bu] 2 66013 Make uploaded files retain modtime with non-modtime backends (Nick 66014 Craig-Wood) 66015 .IP \[bu] 2 66016 Fix incorrect modtime on fs which don\[aq]t support setting modtime 66017 (Nick Craig-Wood) 66018 .IP \[bu] 2 66019 Fix rename of directory containing files to be uploaded (Nick 66020 Craig-Wood) 66021 .RE 66022 .IP \[bu] 2 66023 Local 66024 .RS 2 66025 .IP \[bu] 2 66026 Fix \f[C]%!w(<nil>)\f[R] in \[dq]failed to read directory\[dq] error 66027 (Marks Polakovs) 66028 .IP \[bu] 2 66029 Fix exclusion of dangling symlinks with -L/--copy-links (Nick 66030 Craig-Wood) 66031 .RE 66032 .IP \[bu] 2 66033 Crypt 66034 .RS 2 66035 .IP \[bu] 2 66036 Obey \f[C]--ignore-checksum\f[R] (Nick Craig-Wood) 66037 .IP \[bu] 2 66038 Fix for unencrypted directory names on case insensitive remotes (Ole 66039 Frost) 66040 .RE 66041 .IP \[bu] 2 66042 Azure Blob 66043 .RS 2 66044 .IP \[bu] 2 66045 Remove workarounds for SDK bugs after v0.6.1 update (Nick Craig-Wood) 66046 .RE 66047 .IP \[bu] 2 66048 B2 66049 .RS 2 66050 .IP \[bu] 2 66051 Fix uploading files bigger than 1TiB (Nick Craig-Wood) 66052 .RE 66053 .IP \[bu] 2 66054 Drive 66055 .RS 2 66056 .IP \[bu] 2 66057 Note that \f[C]--drive-acknowledge-abuse\f[R] needs SA Manager 66058 permission (Nick Craig-Wood) 66059 .IP \[bu] 2 66060 Make \f[C]--drive-stop-on-upload-limit\f[R] to respond to 66061 storageQuotaExceeded (Ninh Pham) 66062 .RE 66063 .IP \[bu] 2 66064 FTP 66065 .RS 2 66066 .IP \[bu] 2 66067 Retry 426 errors (Nick Craig-Wood) 66068 .IP \[bu] 2 66069 Retry errors when initiating downloads (Nick Craig-Wood) 66070 .IP \[bu] 2 66071 Revert to upstream \f[C]github.com/jlaffaye/ftp\f[R] now fix is merged 66072 (Nick Craig-Wood) 66073 .RE 66074 .IP \[bu] 2 66075 Google Cloud Storage 66076 .RS 2 66077 .IP \[bu] 2 66078 Add \f[C]--gcs-env-auth\f[R] to pick up IAM credentials from 66079 env/instance (Peter Brunner) 66080 .RE 66081 .IP \[bu] 2 66082 Mega 66083 .RS 2 66084 .IP \[bu] 2 66085 Add \f[C]--mega-use-https\f[R] flag (NodudeWasTaken) 66086 .RE 66087 .IP \[bu] 2 66088 Onedrive 66089 .RS 2 66090 .IP \[bu] 2 66091 Default onedrive personal to QuickXorHash as Microsoft is removing SHA1 66092 (Nick Craig-Wood) 66093 .IP \[bu] 2 66094 Add \f[C]--onedrive-hash-type\f[R] to change the hash in use (Nick 66095 Craig-Wood) 66096 .IP \[bu] 2 66097 Improve speed of QuickXorHash (LXY) 66098 .RE 66099 .IP \[bu] 2 66100 Oracle Object Storage 66101 .RS 2 66102 .IP \[bu] 2 66103 Speed up operations by using S3 pacer and setting minsleep to 10ms 66104 (Manoj Ghosh) 66105 .IP \[bu] 2 66106 Expose the \f[C]storage_tier\f[R] option in config (Manoj Ghosh) 66107 .IP \[bu] 2 66108 Bring your own encryption keys (Manoj Ghosh) 66109 .RE 66110 .IP \[bu] 2 66111 S3 66112 .RS 2 66113 .IP \[bu] 2 66114 Check multipart upload ETag when \f[C]--s3-no-head\f[R] is in use (Nick 66115 Craig-Wood) 66116 .IP \[bu] 2 66117 Add \f[C]--s3-sts-endpoint\f[R] to specify STS endpoint (Nick 66118 Craig-Wood) 66119 .IP \[bu] 2 66120 Fix incorrect tier support for StorJ and IDrive when pointing at a file 66121 (Ole Frost) 66122 .IP \[bu] 2 66123 Fix AWS STS failing if \f[C]--s3-endpoint\f[R] is set (Nick Craig-Wood) 66124 .IP \[bu] 2 66125 Make purge remove directory markers too (Nick Craig-Wood) 66126 .RE 66127 .IP \[bu] 2 66128 Seafile 66129 .RS 2 66130 .IP \[bu] 2 66131 Renew library password (Fred) 66132 .RE 66133 .IP \[bu] 2 66134 SFTP 66135 .RS 2 66136 .IP \[bu] 2 66137 Fix uploads being 65% slower than they should be with crypt (Nick 66138 Craig-Wood) 66139 .RE 66140 .IP \[bu] 2 66141 Smb 66142 .RS 2 66143 .IP \[bu] 2 66144 Allow SPN (service principal name) to be configured (Nick Craig-Wood) 66145 .IP \[bu] 2 66146 Check smb connection is closed (happyxhw) 66147 .RE 66148 .IP \[bu] 2 66149 Storj 66150 .RS 2 66151 .IP \[bu] 2 66152 Implement \f[C]rclone link\f[R] (Kaloyan Raev) 66153 .IP \[bu] 2 66154 Implement \f[C]rclone purge\f[R] (Kaloyan Raev) 66155 .IP \[bu] 2 66156 Update satellite urls and labels (Kaloyan Raev) 66157 .RE 66158 .IP \[bu] 2 66159 WebDAV 66160 .RS 2 66161 .IP \[bu] 2 66162 Fix interop with davrods server (Nick Craig-Wood) 66163 .RE 66164 .SS v1.61.1 - 2022-12-23 66165 .PP 66166 See commits (https://github.com/artpar/artpar/compare/v1.61.0...v1.61.1) 66167 .IP \[bu] 2 66168 Bug Fixes 66169 .RS 2 66170 .IP \[bu] 2 66171 docs: 66172 .RS 2 66173 .IP \[bu] 2 66174 Show only significant parts of version number in version introduced 66175 label (albertony) 66176 .IP \[bu] 2 66177 Fix unescaped HTML (Nick Craig-Wood) 66178 .RE 66179 .IP \[bu] 2 66180 lib/http: Shutdown all servers on exit to remove unix socket (Nick 66181 Craig-Wood) 66182 .IP \[bu] 2 66183 rc: Fix \f[C]--rc-addr\f[R] flag (which is an alternate for 66184 \f[C]--url\f[R]) (Anagh Kumar Baranwal) 66185 .IP \[bu] 2 66186 serve restic 66187 .RS 2 66188 .IP \[bu] 2 66189 Don\[aq]t serve via http if serving via \f[C]--stdio\f[R] (Nick 66190 Craig-Wood) 66191 .IP \[bu] 2 66192 Fix immediate exit when not using stdio (Nick Craig-Wood) 66193 .RE 66194 .IP \[bu] 2 66195 serve webdav 66196 .RS 2 66197 .IP \[bu] 2 66198 Fix \f[C]--baseurl\f[R] handling after \f[C]lib/http\f[R] refactor (Nick 66199 Craig-Wood) 66200 .IP \[bu] 2 66201 Fix running duplicate Serve call (Nick Craig-Wood) 66202 .RE 66203 .RE 66204 .IP \[bu] 2 66205 Azure Blob 66206 .RS 2 66207 .IP \[bu] 2 66208 Fix \[dq]409 Public access is not permitted on this storage account\[dq] 66209 (Nick Craig-Wood) 66210 .RE 66211 .IP \[bu] 2 66212 S3 66213 .RS 2 66214 .IP \[bu] 2 66215 storj: Update endpoints (Kaloyan Raev) 66216 .RE 66217 .SS v1.61.0 - 2022-12-20 66218 .PP 66219 See commits (https://github.com/artpar/artpar/compare/v1.60.0...v1.61.0) 66220 .IP \[bu] 2 66221 New backends 66222 .RS 2 66223 .IP \[bu] 2 66224 New S3 providers 66225 .RS 2 66226 .IP \[bu] 2 66227 Liara LOS (https://rclone.org/s3/#liara-cloud) (MohammadReza) 66228 .RE 66229 .RE 66230 .IP \[bu] 2 66231 New Features 66232 .RS 2 66233 .IP \[bu] 2 66234 build: Add vulnerability testing using govulncheck (albertony) 66235 .IP \[bu] 2 66236 cmd: Enable \f[C]SIGINFO\f[R] (Ctrl-T) handler on FreeBSD, NetBSD, 66237 OpenBSD and Dragonfly BSD (x3-apptech) 66238 .IP \[bu] 2 66239 config: Add config/setpath (https://rclone.org/rc/#config-setpath) for 66240 setting config path via rc/librclone (Nick Craig-Wood) 66241 .IP \[bu] 2 66242 dedupe 66243 .RS 2 66244 .IP \[bu] 2 66245 Count Checks in the stats while scanning for duplicates (Nick 66246 Craig-Wood) 66247 .IP \[bu] 2 66248 Make dedupe obey the filters (Nick Craig-Wood) 66249 .RE 66250 .IP \[bu] 2 66251 dlna: Properly attribute code used from https://github.com/anacrolix/dms 66252 (Nick Craig-Wood) 66253 .IP \[bu] 2 66254 docs 66255 .RS 2 66256 .IP \[bu] 2 66257 Add minimum versions and status badges to backend and command docs (Nick 66258 Craig-Wood, albertony) 66259 .IP \[bu] 2 66260 Remote names may not start or end with space (albertony) 66261 .RE 66262 .IP \[bu] 2 66263 filter: Add metadata filters 66264 --metadata-include/exclude/filter (https://rclone.org/filtering/#metadata) 66265 and friends (Nick Craig-Wood) 66266 .IP \[bu] 2 66267 fs 66268 .RS 2 66269 .IP \[bu] 2 66270 Make all duration flags take \f[C]y\f[R], \f[C]M\f[R], \f[C]w\f[R], 66271 \f[C]d\f[R] etc suffixes (Nick Craig-Wood) 66272 .IP \[bu] 2 66273 Add global flag \f[C]--color\f[R] to control terminal colors (Kevin 66274 Verstaen) 66275 .RE 66276 .IP \[bu] 2 66277 fspath: Allow unicode numbers and letters in remote names (albertony) 66278 .IP \[bu] 2 66279 lib/file: Improve error message for creating dir on non-existent network 66280 host on windows (albertony) 66281 .IP \[bu] 2 66282 lib/http: Finish port of rclone servers to \f[C]lib/http\f[R] (Tom 66283 Mombourquette, Nick Craig-Wood) 66284 .IP \[bu] 2 66285 lib/oauthutil: Improved usability of config flows needing web browser 66286 (Ole Frost) 66287 .IP \[bu] 2 66288 ncdu 66289 .RS 2 66290 .IP \[bu] 2 66291 Add support for modification time (albertony) 66292 .IP \[bu] 2 66293 Fallback to sort by name also for sort by average size (albertony) 66294 .IP \[bu] 2 66295 Rework to use tcell directly instead of the termbox wrapper (eNV25) 66296 .RE 66297 .IP \[bu] 2 66298 rc: Add commands to set GC 66299 Percent (https://rclone.org/rc/#debug-set-gc-percent) & Memory Limit (go 66300 1.19+) (Anagh Kumar Baranwal) 66301 .IP \[bu] 2 66302 rcat: Preserve metadata when Copy falls back to Rcat (Nick Craig-Wood) 66303 .IP \[bu] 2 66304 rcd: Refactor rclone rc server to use \f[C]lib/http\f[R] (Nick 66305 Craig-Wood) 66306 .IP \[bu] 2 66307 rcserver: Avoid generating default credentials with htpasswd (Kamui) 66308 .IP \[bu] 2 66309 restic: Refactor to use \f[C]lib/http\f[R] (Nolan Woods) 66310 .IP \[bu] 2 66311 serve http: Support unix sockets and multiple listeners (Tom 66312 Mombourquette) 66313 .IP \[bu] 2 66314 serve webdav: Refactor to use \f[C]lib/http\f[R] (Nick Craig-Wood) 66315 .IP \[bu] 2 66316 test: Replace defer cleanup with \f[C]t.Cleanup\f[R] (Eng Zer Jun) 66317 .IP \[bu] 2 66318 test memory: Read metadata if \f[C]-M\f[R] flag is specified (Nick 66319 Craig-Wood) 66320 .IP \[bu] 2 66321 wasm: Comply with \f[C]wasm_exec.js\f[R] licence terms (Matthew Vernon) 66322 .RE 66323 .IP \[bu] 2 66324 Bug Fixes 66325 .RS 2 66326 .IP \[bu] 2 66327 build: Update \f[C]golang.org/x/net/http2\f[R] to fix GO-2022-1144 (Nick 66328 Craig-Wood) 66329 .IP \[bu] 2 66330 restic: Fix typo in docs \[aq]remove\[aq] should be \[aq]remote\[aq] 66331 (asdffdsazqqq) 66332 .IP \[bu] 2 66333 serve dlna: Fix panic: Logger uninitialized. 66334 (Nick Craig-Wood) 66335 .RE 66336 .IP \[bu] 2 66337 Mount 66338 .RS 2 66339 .IP \[bu] 2 66340 Update cgofuse for FUSE-T support for mounting volumes on Mac (Nick 66341 Craig-Wood) 66342 .RE 66343 .IP \[bu] 2 66344 VFS 66345 .RS 2 66346 .IP \[bu] 2 66347 Windows: fix slow opening of exe files by not truncating files when not 66348 necessary (Nick Craig-Wood) 66349 .IP \[bu] 2 66350 Fix IO Error opening a file with \f[C]O_CREATE|O_RDONLY\f[R] in 66351 \f[C]--vfs-cache-mode\f[R] not full (Nick Craig-Wood) 66352 .RE 66353 .IP \[bu] 2 66354 Crypt 66355 .RS 2 66356 .IP \[bu] 2 66357 Fix compress wrapping crypt giving upload errors (Nick Craig-Wood) 66358 .RE 66359 .IP \[bu] 2 66360 Azure Blob 66361 .RS 2 66362 .IP \[bu] 2 66363 Port to new SDK (Nick Craig-Wood) 66364 .RS 2 66365 .IP \[bu] 2 66366 Revamp authentication to include all methods and docs (Nick Craig-Wood) 66367 .IP \[bu] 2 66368 Port old authentication methods to new SDK (Nick Craig-Wood, Brad 66369 Ackerman) 66370 .IP \[bu] 2 66371 Thanks to Stonebranch (https://www.stonebranch.com/) for sponsoring this 66372 work. 66373 .RE 66374 .IP \[bu] 2 66375 Add \f[C]--azureblob-no-check-container\f[R] to assume container exists 66376 (Nick Craig-Wood) 66377 .IP \[bu] 2 66378 Add \f[C]--use-server-modtime\f[R] support (Abdullah Saglam) 66379 .IP \[bu] 2 66380 Add support for custom upload headers (rkettelerij) 66381 .IP \[bu] 2 66382 Allow emulator account/key override (Roel Arents) 66383 .IP \[bu] 2 66384 Support simple \[dq]environment credentials\[dq] (Nathaniel Wesley 66385 Filardo) 66386 .IP \[bu] 2 66387 Ignore \f[C]AuthorizationFailure\f[R] when trying to create a create a 66388 container (Nick Craig-Wood) 66389 .RE 66390 .IP \[bu] 2 66391 Box 66392 .RS 2 66393 .IP \[bu] 2 66394 Added note on Box API rate limits (Ole Frost) 66395 .RE 66396 .IP \[bu] 2 66397 Drive 66398 .RS 2 66399 .IP \[bu] 2 66400 Handle shared drives with leading/trailing space in name (related to) 66401 (albertony) 66402 .RE 66403 .IP \[bu] 2 66404 FTP 66405 .RS 2 66406 .IP \[bu] 2 66407 Update help text of implicit/explicit TLS options to refer to FTPS 66408 instead of FTP (ycdtosa) 66409 .IP \[bu] 2 66410 Improve performance to speed up \f[C]--files-from\f[R] and 66411 \f[C]NewObject\f[R] (Anthony Pessy) 66412 .RE 66413 .IP \[bu] 2 66414 HTTP 66415 .RS 2 66416 .IP \[bu] 2 66417 Parse GET responses when \f[C]no_head\f[R] is set (Arnie97) 66418 .IP \[bu] 2 66419 Do not update object size based on \f[C]Range\f[R] requests (Arnie97) 66420 .IP \[bu] 2 66421 Support \f[C]Content-Range\f[R] response header (Arnie97) 66422 .RE 66423 .IP \[bu] 2 66424 Onedrive 66425 .RS 2 66426 .IP \[bu] 2 66427 Document workaround for shared with me files (vanplus) 66428 .RE 66429 .IP \[bu] 2 66430 S3 66431 .RS 2 66432 .IP \[bu] 2 66433 Add Liara LOS to provider list (MohammadReza) 66434 .IP \[bu] 2 66435 Add DigitalOcean Spaces regions \f[C]sfo3\f[R], \f[C]fra1\f[R], 66436 \f[C]syd1\f[R] (Jack) 66437 .IP \[bu] 2 66438 Avoid privileged \f[C]GetBucketLocation\f[R] to resolve s3 region 66439 (Anthony Pessy) 66440 .IP \[bu] 2 66441 Stop setting object and bucket ACL to \f[C]private\f[R] if it is an 66442 empty string (Philip Harvey) 66443 .IP \[bu] 2 66444 If bucket or object ACL is empty string then don\[aq]t add 66445 \f[C]X-Amz-Acl:\f[R] header (Nick Craig-Wood) 66446 .IP \[bu] 2 66447 Reduce memory consumption for s3 objects (Erik Agterdenbos) 66448 .IP \[bu] 2 66449 Fix listing loop when using v2 listing on v1 server (Nick Craig-Wood) 66450 .IP \[bu] 2 66451 Fix nil pointer exception when using Versions (Nick Craig-Wood) 66452 .IP \[bu] 2 66453 Fix excess memory usage when using versions (Nick Craig-Wood) 66454 .IP \[bu] 2 66455 Ignore versionIDs from uploads unless using \f[C]--s3-versions\f[R] or 66456 \f[C]--s3-versions-at\f[R] (Nick Craig-Wood) 66457 .RE 66458 .IP \[bu] 2 66459 SFTP 66460 .RS 2 66461 .IP \[bu] 2 66462 Add configuration options to set ssh Ciphers / MACs / KeyExchange 66463 (dgouju) 66464 .IP \[bu] 2 66465 Auto-detect shell type for fish (albertony) 66466 .IP \[bu] 2 66467 Fix NewObject with leading / (Nick Craig-Wood) 66468 .RE 66469 .IP \[bu] 2 66470 Smb 66471 .RS 2 66472 .IP \[bu] 2 66473 Fix issue where spurious dot directory is created (albertony) 66474 .RE 66475 .IP \[bu] 2 66476 Storj 66477 .RS 2 66478 .IP \[bu] 2 66479 Implement server side Copy (Kaloyan Raev) 66480 .RE 66481 .SS v1.60.1 - 2022-11-17 66482 .PP 66483 See commits (https://github.com/artpar/artpar/compare/v1.60.0...v1.60.1) 66484 .IP \[bu] 2 66485 Bug Fixes 66486 .RS 2 66487 .IP \[bu] 2 66488 lib/cache: Fix alias backend shutting down too soon (Nick Craig-Wood) 66489 .IP \[bu] 2 66490 wasm: Fix walltime link error by adding up-to-date wasm_exec.js 66491 (Jo\[~a]o Henrique Franco) 66492 .IP \[bu] 2 66493 docs 66494 .RS 2 66495 .IP \[bu] 2 66496 Update faq.md with bisync (Samuel Johnson) 66497 .IP \[bu] 2 66498 Corrected download links in windows install docs (coultonluke) 66499 .IP \[bu] 2 66500 Add direct download link for windows arm64 (albertony) 66501 .IP \[bu] 2 66502 Remove link to rclone slack as it is no longer supported (Nick 66503 Craig-Wood) 66504 .IP \[bu] 2 66505 Faq: how to use a proxy server that requires a username and password 66506 (asdffdsazqqq) 66507 .IP \[bu] 2 66508 Oracle-object-storage: doc fix (Manoj Ghosh) 66509 .IP \[bu] 2 66510 Fix typo \f[C]remove\f[R] in rclone_serve_restic command (Joda 66511 St\[:o]\[ss]er) 66512 .IP \[bu] 2 66513 Fix character that was incorrectly interpreted as markdown (Cl\['e]ment 66514 Notin) 66515 .RE 66516 .RE 66517 .IP \[bu] 2 66518 VFS 66519 .RS 2 66520 .IP \[bu] 2 66521 Fix deadlock caused by cache cleaner and upload finishing (Nick 66522 Craig-Wood) 66523 .RE 66524 .IP \[bu] 2 66525 Local 66526 .RS 2 66527 .IP \[bu] 2 66528 Clean absolute paths (albertony) 66529 .IP \[bu] 2 66530 Fix -L/--copy-links with filters missing directories (Nick Craig-Wood) 66531 .RE 66532 .IP \[bu] 2 66533 Mailru 66534 .RS 2 66535 .IP \[bu] 2 66536 Note that an app password is now needed (Nick Craig-Wood) 66537 .IP \[bu] 2 66538 Allow timestamps to be before the epoch 1970-01-01 (Nick Craig-Wood) 66539 .RE 66540 .IP \[bu] 2 66541 S3 66542 .RS 2 66543 .IP \[bu] 2 66544 Add provider quirk \f[C]--s3-might-gzip\f[R] to fix corrupted on 66545 transfer: sizes differ (Nick Craig-Wood) 66546 .IP \[bu] 2 66547 Allow Storj to server side copy since it seems to work now (Nick 66548 Craig-Wood) 66549 .IP \[bu] 2 66550 Fix for unchecked err value in s3 listv2 (Aaron Gokaslan) 66551 .IP \[bu] 2 66552 Add additional Wasabi locations (techknowlogick) 66553 .RE 66554 .IP \[bu] 2 66555 Smb 66556 .RS 2 66557 .IP \[bu] 2 66558 Fix \f[C]Failed to sync: context canceled\f[R] at the end of syncs (Nick 66559 Craig-Wood) 66560 .RE 66561 .IP \[bu] 2 66562 WebDAV 66563 .RS 2 66564 .IP \[bu] 2 66565 Fix Move/Copy/DirMove when using -server-side-across-configs (Nick 66566 Craig-Wood) 66567 .RE 66568 .SS v1.60.0 - 2022-10-21 66569 .PP 66570 See commits (https://github.com/artpar/artpar/compare/v1.59.0...v1.60.0) 66571 .IP \[bu] 2 66572 New backends 66573 .RS 2 66574 .IP \[bu] 2 66575 Oracle object storage (https://rclone.org/oracleobjectstorage/) (Manoj 66576 Ghosh) 66577 .IP \[bu] 2 66578 SMB (https://rclone.org/smb/) / CIFS (Windows file sharing) (Lesmiscore) 66579 .IP \[bu] 2 66580 New S3 providers 66581 .RS 2 66582 .IP \[bu] 2 66583 IONOS Cloud Storage (https://rclone.org/s3/#ionos) (Dmitry Deniskin) 66584 .IP \[bu] 2 66585 Qiniu KODO (https://rclone.org/s3/#qiniu) (Bachue Zhou) 66586 .RE 66587 .RE 66588 .IP \[bu] 2 66589 New Features 66590 .RS 2 66591 .IP \[bu] 2 66592 build 66593 .RS 2 66594 .IP \[bu] 2 66595 Update to go1.19 and make go1.17 the minimum required version (Nick 66596 Craig-Wood) 66597 .IP \[bu] 2 66598 Install.sh: fix arm-v7 download (Ole Frost) 66599 .RE 66600 .IP \[bu] 2 66601 fs: Warn the user when using an existing remote name without a colon 66602 (Nick Craig-Wood) 66603 .IP \[bu] 2 66604 httplib: Add \f[C]--xxx-min-tls-version\f[R] option to select minimum 66605 TLS version for HTTP servers (Robert Newson) 66606 .IP \[bu] 2 66607 librclone: Add PHP bindings and test program (Jordi Gonzalez Mu\[~n]oz) 66608 .IP \[bu] 2 66609 operations 66610 .RS 2 66611 .IP \[bu] 2 66612 Add \f[C]--server-side-across-configs\f[R] global flag for any backend 66613 (Nick Craig-Wood) 66614 .IP \[bu] 2 66615 Optimise \f[C]--copy-dest\f[R] and \f[C]--compare-dest\f[R] (Nick 66616 Craig-Wood) 66617 .RE 66618 .IP \[bu] 2 66619 rc: add \f[C]job/stopgroup\f[R] to stop group (Evan Spensley) 66620 .IP \[bu] 2 66621 serve dlna 66622 .RS 2 66623 .IP \[bu] 2 66624 Add \f[C]--announce-interval\f[R] to control SSDP Announce Interval 66625 (YanceyChiew) 66626 .IP \[bu] 2 66627 Add \f[C]--interface\f[R] to Specify SSDP interface names line (Simon 66628 Bos) 66629 .IP \[bu] 2 66630 Add support for more external subtitles (YanceyChiew) 66631 .IP \[bu] 2 66632 Add verification of addresses (YanceyChiew) 66633 .RE 66634 .IP \[bu] 2 66635 sync: Optimise \f[C]--copy-dest\f[R] and \f[C]--compare-dest\f[R] (Nick 66636 Craig-Wood) 66637 .IP \[bu] 2 66638 doc updates (albertony, Alexander Knorr, anonion, Jo\[~a]o Henrique 66639 Franco, Josh Soref, Lorenzo Milesi, Marco Molteni, Mark Trolley, Ole 66640 Frost, partev, Ryan Morey, Tom Mombourquette, YFdyh000) 66641 .RE 66642 .IP \[bu] 2 66643 Bug Fixes 66644 .RS 2 66645 .IP \[bu] 2 66646 filter 66647 .RS 2 66648 .IP \[bu] 2 66649 Fix incorrect filtering with \f[C]UseFilter\f[R] context flag and 66650 wrapping backends (Nick Craig-Wood) 66651 .IP \[bu] 2 66652 Make sure we check \f[C]--files-from\f[R] when looking for a single file 66653 (Nick Craig-Wood) 66654 .RE 66655 .IP \[bu] 2 66656 rc 66657 .RS 2 66658 .IP \[bu] 2 66659 Fix \f[C]mount/listmounts\f[R] not returning the full Fs entered in 66660 \f[C]mount/mount\f[R] (Tom Mombourquette) 66661 .IP \[bu] 2 66662 Handle external unmount when mounting (Isaac Aymerich) 66663 .IP \[bu] 2 66664 Validate Daemon option is not set when mounting a volume via RC (Isaac 66665 Aymerich) 66666 .RE 66667 .IP \[bu] 2 66668 sync: Update docs and error messages to reflect fixes to overlap checks 66669 (Nick Naumann) 66670 .RE 66671 .IP \[bu] 2 66672 VFS 66673 .RS 2 66674 .IP \[bu] 2 66675 Reduce memory use by embedding \f[C]sync.Cond\f[R] (Nick Craig-Wood) 66676 .IP \[bu] 2 66677 Reduce memory usage by re-ordering commonly used structures (Nick 66678 Craig-Wood) 66679 .IP \[bu] 2 66680 Fix excess CPU used by VFS cache cleaner looping (Nick Craig-Wood) 66681 .RE 66682 .IP \[bu] 2 66683 Local 66684 .RS 2 66685 .IP \[bu] 2 66686 Obey file filters in listing to fix errors on excluded files (Nick 66687 Craig-Wood) 66688 .IP \[bu] 2 66689 Fix \[dq]Failed to read metadata: function not implemented\[dq] on old 66690 Linux kernels (Nick Craig-Wood) 66691 .RE 66692 .IP \[bu] 2 66693 Compress 66694 .RS 2 66695 .IP \[bu] 2 66696 Fix crash due to nil metadata (Nick Craig-Wood) 66697 .IP \[bu] 2 66698 Fix error handling to not use or return nil objects (Nick Craig-Wood) 66699 .RE 66700 .IP \[bu] 2 66701 Drive 66702 .RS 2 66703 .IP \[bu] 2 66704 Make \f[C]--drive-stop-on-upload-limit\f[R] obey quota exceeded error 66705 (Steve Kowalik) 66706 .RE 66707 .IP \[bu] 2 66708 FTP 66709 .RS 2 66710 .IP \[bu] 2 66711 Add \f[C]--ftp-force-list-hidden\f[R] option to show hidden items 66712 (\[/O]yvind Heddeland Instefjord) 66713 .IP \[bu] 2 66714 Fix hang when using ExplicitTLS to certain servers. 66715 (Nick Craig-Wood) 66716 .RE 66717 .IP \[bu] 2 66718 Google Cloud Storage 66719 .RS 2 66720 .IP \[bu] 2 66721 Add \f[C]--gcs-endpoint\f[R] flag and config parameter (Nick Craig-Wood) 66722 .RE 66723 .IP \[bu] 2 66724 Hubic 66725 .RS 2 66726 .IP \[bu] 2 66727 Remove backend as service has now shut down (Nick Craig-Wood) 66728 .RE 66729 .IP \[bu] 2 66730 Onedrive 66731 .RS 2 66732 .IP \[bu] 2 66733 Rename Onedrive(cn) 21Vianet to Vnet Group (Yen Hu) 66734 .IP \[bu] 2 66735 Disable change notify in China region since it is not supported (Nick 66736 Craig-Wood) 66737 .RE 66738 .IP \[bu] 2 66739 S3 66740 .RS 2 66741 .IP \[bu] 2 66742 Implement \f[C]--s3-versions\f[R] flag to show old versions of objects 66743 if enabled (Nick Craig-Wood) 66744 .IP \[bu] 2 66745 Implement \f[C]--s3-version-at\f[R] flag to show versions of objects at 66746 a particular time (Nick Craig-Wood) 66747 .IP \[bu] 2 66748 Implement \f[C]backend versioning\f[R] command to get/set bucket 66749 versioning (Nick Craig-Wood) 66750 .IP \[bu] 2 66751 Implement \f[C]Purge\f[R] to purge versions and 66752 \f[C]backend cleanup-hidden\f[R] (Nick Craig-Wood) 66753 .IP \[bu] 2 66754 Add \f[C]--s3-decompress\f[R] flag to decompress gzip-encoded files 66755 (Nick Craig-Wood) 66756 .IP \[bu] 2 66757 Add \f[C]--s3-sse-customer-key-base64\f[R] to supply keys with binary 66758 data (Richard Bateman) 66759 .IP \[bu] 2 66760 Try to keep the maximum precision in ModTime with 66761 \f[C]--user-server-modtime\f[R] (Nick Craig-Wood) 66762 .IP \[bu] 2 66763 Drop binary metadata with an ERROR message as it can\[aq]t be stored 66764 (Nick Craig-Wood) 66765 .IP \[bu] 2 66766 Add \f[C]--s3-no-system-metadata\f[R] to suppress read and write of 66767 system metadata (Nick Craig-Wood) 66768 .RE 66769 .IP \[bu] 2 66770 SFTP 66771 .RS 2 66772 .IP \[bu] 2 66773 Fix directory creation races (Lesmiscore) 66774 .RE 66775 .IP \[bu] 2 66776 Swift 66777 .RS 2 66778 .IP \[bu] 2 66779 Add \f[C]--swift-no-large-objects\f[R] to reduce HEAD requests (Nick 66780 Craig-Wood) 66781 .RE 66782 .IP \[bu] 2 66783 Union 66784 .RS 2 66785 .IP \[bu] 2 66786 Propagate SlowHash feature to fix hasher interaction (Lesmiscore) 66787 .RE 66788 .SS v1.59.2 - 2022-09-15 66789 .PP 66790 See commits (https://github.com/artpar/artpar/compare/v1.59.1...v1.59.2) 66791 .IP \[bu] 2 66792 Bug Fixes 66793 .RS 2 66794 .IP \[bu] 2 66795 config: Move locking to fix fatal error: concurrent map read and map 66796 write (Nick Craig-Wood) 66797 .RE 66798 .IP \[bu] 2 66799 Local 66800 .RS 2 66801 .IP \[bu] 2 66802 Disable xattr support if the filesystems indicates it is not supported 66803 (Nick Craig-Wood) 66804 .RE 66805 .IP \[bu] 2 66806 Azure Blob 66807 .RS 2 66808 .IP \[bu] 2 66809 Fix chunksize calculations producing too many parts (Nick Craig-Wood) 66810 .RE 66811 .IP \[bu] 2 66812 B2 66813 .RS 2 66814 .IP \[bu] 2 66815 Fix chunksize calculations producing too many parts (Nick Craig-Wood) 66816 .RE 66817 .IP \[bu] 2 66818 S3 66819 .RS 2 66820 .IP \[bu] 2 66821 Fix chunksize calculations producing too many parts (Nick Craig-Wood) 66822 .RE 66823 .SS v1.59.1 - 2022-08-08 66824 .PP 66825 See commits (https://github.com/artpar/artpar/compare/v1.59.0...v1.59.1) 66826 .IP \[bu] 2 66827 Bug Fixes 66828 .RS 2 66829 .IP \[bu] 2 66830 accounting: Fix panic in core/stats-reset with unknown group (Nick 66831 Craig-Wood) 66832 .IP \[bu] 2 66833 build: Fix android build after GitHub actions change (Nick Craig-Wood) 66834 .IP \[bu] 2 66835 dlna: Fix SOAP action header parsing (Joram Schrijver) 66836 .IP \[bu] 2 66837 docs: Fix links to mount command from install docs (albertony) 66838 .IP \[bu] 2 66839 dropbox: Fix ChangeNotify was unable to decrypt errors (Nick Craig-Wood) 66840 .IP \[bu] 2 66841 fs: Fix parsing of times and durations of the form \[dq]YYYY-MM-DD 66842 HH:MM:SS\[dq] (Nick Craig-Wood) 66843 .IP \[bu] 2 66844 serve sftp: Fix checksum detection (Nick Craig-Wood) 66845 .IP \[bu] 2 66846 sync: Add accidentally missed filter-sensitivity to --backup-dir option 66847 (Nick Naumann) 66848 .RE 66849 .IP \[bu] 2 66850 Combine 66851 .RS 2 66852 .IP \[bu] 2 66853 Fix docs showing \f[C]remote=\f[R] instead of \f[C]upstreams=\f[R] (Nick 66854 Craig-Wood) 66855 .IP \[bu] 2 66856 Throw error if duplicate directory name is specified (Nick Craig-Wood) 66857 .IP \[bu] 2 66858 Fix errors with backends shutting down while in use (Nick Craig-Wood) 66859 .RE 66860 .IP \[bu] 2 66861 Dropbox 66862 .RS 2 66863 .IP \[bu] 2 66864 Fix hang on quit with --dropbox-batch-mode off (Nick Craig-Wood) 66865 .IP \[bu] 2 66866 Fix infinite loop on uploading a corrupted file (Nick Craig-Wood) 66867 .RE 66868 .IP \[bu] 2 66869 Internetarchive 66870 .RS 2 66871 .IP \[bu] 2 66872 Ignore checksums for files using the different method (Lesmiscore) 66873 .IP \[bu] 2 66874 Handle hash symbol in the middle of filename (Lesmiscore) 66875 .RE 66876 .IP \[bu] 2 66877 Jottacloud 66878 .RS 2 66879 .IP \[bu] 2 66880 Fix working with whitelabel Elgiganten Cloud 66881 .IP \[bu] 2 66882 Do not store username in config when using standard auth (albertony) 66883 .RE 66884 .IP \[bu] 2 66885 Mega 66886 .RS 2 66887 .IP \[bu] 2 66888 Fix nil pointer exception when bad node received (Nick Craig-Wood) 66889 .RE 66890 .IP \[bu] 2 66891 S3 66892 .RS 2 66893 .IP \[bu] 2 66894 Fix --s3-no-head panic: reflect: Elem of invalid type s3.PutObjectInput 66895 (Nick Craig-Wood) 66896 .RE 66897 .IP \[bu] 2 66898 SFTP 66899 .RS 2 66900 .IP \[bu] 2 66901 Fix issue with WS_FTP by working around failing RealPath (albertony) 66902 .RE 66903 .IP \[bu] 2 66904 Union 66905 .RS 2 66906 .IP \[bu] 2 66907 Fix duplicated files when using directories with leading / (Nick 66908 Craig-Wood) 66909 .IP \[bu] 2 66910 Fix multiple files being uploaded when roots don\[aq]t exist (Nick 66911 Craig-Wood) 66912 .IP \[bu] 2 66913 Fix panic due to misalignment of struct field in 32 bit architectures 66914 (r-ricci) 66915 .RE 66916 .SS v1.59.0 - 2022-07-09 66917 .PP 66918 See commits (https://github.com/artpar/artpar/compare/v1.58.0...v1.59.0) 66919 .IP \[bu] 2 66920 New backends 66921 .RS 2 66922 .IP \[bu] 2 66923 Combine multiple remotes in one directory tree (Nick Craig-Wood) 66924 .IP \[bu] 2 66925 Hidrive (https://rclone.org/hidrive/) (Ovidiu Victor Tatar) 66926 .IP \[bu] 2 66927 Internet Archive (https://rclone.org/internetarchive/) (Lesmiscore 66928 (Naoya Ozaki)) 66929 .IP \[bu] 2 66930 New S3 providers 66931 .RS 2 66932 .IP \[bu] 2 66933 ArvanCloud AOS (https://rclone.org/s3/#arvan-cloud) (ehsantdy) 66934 .IP \[bu] 2 66935 Cloudflare R2 (https://rclone.org/s3/#cloudflare-r2) (Nick Craig-Wood) 66936 .IP \[bu] 2 66937 Huawei OBS (https://rclone.org/s3/#huawei-obs) (m00594701) 66938 .IP \[bu] 2 66939 IDrive e2 (https://rclone.org/s3/#idrive-e2) (vyloy) 66940 .RE 66941 .RE 66942 .IP \[bu] 2 66943 New commands 66944 .RS 2 66945 .IP \[bu] 2 66946 test makefile (https://rclone.org/commands/rclone_test_makefile/): 66947 Create a single file for testing (Nick Craig-Wood) 66948 .RE 66949 .IP \[bu] 2 66950 New Features 66951 .RS 2 66952 .IP \[bu] 2 66953 Metadata framework (https://rclone.org/docs/#metadata) to read and write 66954 system and user metadata on backends (Nick Craig-Wood) 66955 .RS 2 66956 .IP \[bu] 2 66957 Implemented initially for \f[C]local\f[R], \f[C]s3\f[R] and 66958 \f[C]internetarchive\f[R] backends 66959 .IP \[bu] 2 66960 \f[C]--metadata\f[R]/\f[C]-M\f[R] flag to control whether metadata is 66961 copied 66962 .IP \[bu] 2 66963 \f[C]--metadata-set\f[R] flag to specify metadata for uploads 66964 .IP \[bu] 2 66965 Thanks to Manz Solutions (https://manz-solutions.at/) for sponsoring 66966 this work. 66967 .RE 66968 .IP \[bu] 2 66969 build 66970 .RS 2 66971 .IP \[bu] 2 66972 Update to go1.18 and make go1.16 the minimum required version (Nick 66973 Craig-Wood) 66974 .IP \[bu] 2 66975 Update android go build to 1.18.x and NDK to 23.1.7779620 (Nick 66976 Craig-Wood) 66977 .IP \[bu] 2 66978 All windows binaries now no longer CGO (Nick Craig-Wood) 66979 .IP \[bu] 2 66980 Add \f[C]linux/arm/v6\f[R] to docker images (Nick Craig-Wood) 66981 .IP \[bu] 2 66982 A huge number of fixes found with staticcheck (https://staticcheck.io/) 66983 (albertony) 66984 .IP \[bu] 2 66985 Configurable version suffix independent of version number (albertony) 66986 .RE 66987 .IP \[bu] 2 66988 check: Implement \f[C]--no-traverse\f[R] and 66989 \f[C]--no-unicode-normalization\f[R] (Nick Craig-Wood) 66990 .IP \[bu] 2 66991 config: Readability improvements (albertony) 66992 .IP \[bu] 2 66993 copyurl: Add \f[C]--header-filename\f[R] to honor the HTTP header 66994 filename directive (J-P Treen) 66995 .IP \[bu] 2 66996 filter: Allow multiple \f[C]--exclude-if-present\f[R] flags (albertony) 66997 .IP \[bu] 2 66998 fshttp: Add \f[C]--disable-http-keep-alives\f[R] to disable HTTP Keep 66999 Alives (Nick Craig-Wood) 67000 .IP \[bu] 2 67001 install.sh 67002 .RS 2 67003 .IP \[bu] 2 67004 Set the modes on the files and/or directories on macOS (Michael C 67005 Tiernan - MIT-Research Computing Project) 67006 .IP \[bu] 2 67007 Pre verify sudo authorization \f[C]-v\f[R] before calling curl. 67008 (Michael C Tiernan - MIT-Research Computing Project) 67009 .RE 67010 .IP \[bu] 2 67011 lib/encoder: Add Semicolon encoding (Nick Craig-Wood) 67012 .IP \[bu] 2 67013 lsf: Add metadata support with \f[C]M\f[R] flag (Nick Craig-Wood) 67014 .IP \[bu] 2 67015 lsjson: Add \f[C]--metadata\f[R]/\f[C]-M\f[R] flag (Nick Craig-Wood) 67016 .IP \[bu] 2 67017 ncdu 67018 .RS 2 67019 .IP \[bu] 2 67020 Implement multi selection (CrossR) 67021 .IP \[bu] 2 67022 Replace termbox with tcell\[aq]s termbox wrapper (eNV25) 67023 .IP \[bu] 2 67024 Display correct path in delete confirmation dialog (Roberto Ricci) 67025 .RE 67026 .IP \[bu] 2 67027 operations 67028 .RS 2 67029 .IP \[bu] 2 67030 Speed up hash checking by aborting the other hash if first returns 67031 nothing (Nick Craig-Wood) 67032 .IP \[bu] 2 67033 Use correct src/dst in some log messages (zzr93) 67034 .RE 67035 .IP \[bu] 2 67036 rcat: Check checksums by default like copy does (Nick Craig-Wood) 67037 .IP \[bu] 2 67038 selfupdate: Replace deprecated \f[C]x/crypto/openpgp\f[R] package with 67039 \f[C]ProtonMail/go-crypto\f[R] (albertony) 67040 .IP \[bu] 2 67041 serve ftp: Check \f[C]--passive-port\f[R] arguments are correct (Nick 67042 Craig-Wood) 67043 .IP \[bu] 2 67044 size: Warn about inaccurate results when objects with unknown size 67045 (albertony) 67046 .IP \[bu] 2 67047 sync: Overlap check is now filter-sensitive so \f[C]--backup-dir\f[R] 67048 can be in the root provided it is filtered (Nick) 67049 .IP \[bu] 2 67050 test info: Check file name lengths using 1,2,3,4 byte unicode characters 67051 (Nick Craig-Wood) 67052 .IP \[bu] 2 67053 test makefile(s): \f[C]--sparse\f[R], \f[C]--zero\f[R], 67054 \f[C]--pattern\f[R], \f[C]--ascii\f[R], \f[C]--chargen\f[R] flags to 67055 control file contents (Nick Craig-Wood) 67056 .IP \[bu] 2 67057 Make sure we call the \f[C]Shutdown\f[R] method on backends (Martin 67058 Czygan) 67059 .RE 67060 .IP \[bu] 2 67061 Bug Fixes 67062 .RS 2 67063 .IP \[bu] 2 67064 accounting: Fix unknown length file transfers counting 3 transfers each 67065 (buda) 67066 .IP \[bu] 2 67067 ncdu: Fix issue where dir size is summed when file sizes are -1 67068 (albertony) 67069 .IP \[bu] 2 67070 sync/copy/move 67071 .RS 2 67072 .IP \[bu] 2 67073 Fix \f[C]--fast-list\f[R] \f[C]--create-empty-src-dirs\f[R] and 67074 \f[C]--exclude\f[R] (Nick Craig-Wood) 67075 .IP \[bu] 2 67076 Fix \f[C]--max-duration\f[R] and \f[C]--cutoff-mode soft\f[R] (Nick 67077 Craig-Wood) 67078 .RE 67079 .IP \[bu] 2 67080 Fix fs cache unpin (Martin Czygan) 67081 .IP \[bu] 2 67082 Set proper exit code for errors that are not low-level retried (e.g. 67083 size/timestamp changing) (albertony) 67084 .RE 67085 .IP \[bu] 2 67086 Mount 67087 .RS 2 67088 .IP \[bu] 2 67089 Support \f[C]windows/arm64\f[R] (may still be problems - see 67090 #5828 (https://github.com/artpar/artpar/issues/5828)) (Nick Craig-Wood) 67091 .IP \[bu] 2 67092 Log IO errors at ERROR level (Nick Craig-Wood) 67093 .IP \[bu] 2 67094 Ignore \f[C]_netdev\f[R] mount argument (Hugal31) 67095 .RE 67096 .IP \[bu] 2 67097 VFS 67098 .RS 2 67099 .IP \[bu] 2 67100 Add \f[C]--vfs-fast-fingerprint\f[R] for less accurate but faster 67101 fingerprints (Nick Craig-Wood) 67102 .IP \[bu] 2 67103 Add \f[C]--vfs-disk-space-total-size\f[R] option to manually set the 67104 total disk space (Claudio Maradonna) 67105 .IP \[bu] 2 67106 vfscache: Fix fatal error: sync: unlock of unlocked mutex error (Nick 67107 Craig-Wood) 67108 .RE 67109 .IP \[bu] 2 67110 Local 67111 .RS 2 67112 .IP \[bu] 2 67113 Fix parsing of \f[C]--local-nounc\f[R] flag (Nick Craig-Wood) 67114 .IP \[bu] 2 67115 Add Metadata support (Nick Craig-Wood) 67116 .RE 67117 .IP \[bu] 2 67118 Crypt 67119 .RS 2 67120 .IP \[bu] 2 67121 Support metadata (Nick Craig-Wood) 67122 .RE 67123 .IP \[bu] 2 67124 Azure Blob 67125 .RS 2 67126 .IP \[bu] 2 67127 Calculate Chunksize/blocksize to stay below maxUploadParts (Leroy van 67128 Logchem) 67129 .IP \[bu] 2 67130 Use chunksize lib to determine chunksize dynamically (Derek Battams) 67131 .IP \[bu] 2 67132 Case insensitive access tier (Rob Pickerill) 67133 .IP \[bu] 2 67134 Allow remote emulator (azurite) (Lorenzo Maiorfi) 67135 .RE 67136 .IP \[bu] 2 67137 B2 67138 .RS 2 67139 .IP \[bu] 2 67140 Add \f[C]--b2-version-at\f[R] flag to show file versions at time 67141 specified (SwazRGB) 67142 .IP \[bu] 2 67143 Use chunksize lib to determine chunksize dynamically (Derek Battams) 67144 .RE 67145 .IP \[bu] 2 67146 Chunker 67147 .RS 2 67148 .IP \[bu] 2 67149 Mark as not supporting metadata (Nick Craig-Wood) 67150 .RE 67151 .IP \[bu] 2 67152 Compress 67153 .RS 2 67154 .IP \[bu] 2 67155 Support metadata (Nick Craig-Wood) 67156 .RE 67157 .IP \[bu] 2 67158 Drive 67159 .RS 2 67160 .IP \[bu] 2 67161 Make \f[C]backend config -o config\f[R] add a combined 67162 \f[C]AllDrives:\f[R] remote (Nick Craig-Wood) 67163 .IP \[bu] 2 67164 Make \f[C]--drive-shared-with-me\f[R] work with shared drives (Nick 67165 Craig-Wood) 67166 .IP \[bu] 2 67167 Add \f[C]--drive-resource-key\f[R] for accessing link-shared files (Nick 67168 Craig-Wood) 67169 .IP \[bu] 2 67170 Add backend commands \f[C]exportformats\f[R] and \f[C]importformats\f[R] 67171 for debugging (Nick Craig-Wood) 67172 .IP \[bu] 2 67173 Fix 404 errors on copy/server side copy objects from public folder (Nick 67174 Craig-Wood) 67175 .IP \[bu] 2 67176 Update Internal OAuth consent screen docs (Phil Shackleton) 67177 .IP \[bu] 2 67178 Moved \f[C]root_folder_id\f[R] to advanced section (Abhiraj) 67179 .RE 67180 .IP \[bu] 2 67181 Dropbox 67182 .RS 2 67183 .IP \[bu] 2 67184 Migrate from deprecated api (m8rge) 67185 .IP \[bu] 2 67186 Add logs to show when poll interval limits are exceeded (Nick 67187 Craig-Wood) 67188 .IP \[bu] 2 67189 Fix nil pointer exception on dropbox impersonate user not found (Nick 67190 Craig-Wood) 67191 .RE 67192 .IP \[bu] 2 67193 Fichier 67194 .RS 2 67195 .IP \[bu] 2 67196 Parse api error codes and them accordingly (buengese) 67197 .RE 67198 .IP \[bu] 2 67199 FTP 67200 .RS 2 67201 .IP \[bu] 2 67202 Add support for \f[C]disable_utf8\f[R] option (Jason Zheng) 67203 .IP \[bu] 2 67204 Revert to upstream \f[C]github.com/jlaffaye/ftp\f[R] from our fork (Nick 67205 Craig-Wood) 67206 .RE 67207 .IP \[bu] 2 67208 Google Cloud Storage 67209 .RS 2 67210 .IP \[bu] 2 67211 Add \f[C]--gcs-no-check-bucket\f[R] to minimise transactions and perms 67212 (Nick Gooding) 67213 .IP \[bu] 2 67214 Add \f[C]--gcs-decompress\f[R] flag to decompress gzip-encoded files 67215 (Nick Craig-Wood) 67216 .RS 2 67217 .IP \[bu] 2 67218 by default these will be downloaded compressed (which previously failed) 67219 .RE 67220 .RE 67221 .IP \[bu] 2 67222 Hasher 67223 .RS 2 67224 .IP \[bu] 2 67225 Support metadata (Nick Craig-Wood) 67226 .RE 67227 .IP \[bu] 2 67228 HTTP 67229 .RS 2 67230 .IP \[bu] 2 67231 Fix missing response when using custom auth handler (albertony) 67232 .RE 67233 .IP \[bu] 2 67234 Jottacloud 67235 .RS 2 67236 .IP \[bu] 2 67237 Add support for upload to custom device and mountpoint (albertony) 67238 .IP \[bu] 2 67239 Always store username in config and use it to avoid initial API request 67240 (albertony) 67241 .IP \[bu] 2 67242 Fix issue with server-side copy when destination is in trash (albertony) 67243 .IP \[bu] 2 67244 Fix listing output of remote with special characters (albertony) 67245 .RE 67246 .IP \[bu] 2 67247 Mailru 67248 .RS 2 67249 .IP \[bu] 2 67250 Fix timeout by using int instead of time.Duration for keeping number of 67251 seconds (albertony) 67252 .RE 67253 .IP \[bu] 2 67254 Mega 67255 .RS 2 67256 .IP \[bu] 2 67257 Document using MEGAcmd to help with login failures (Art M. 67258 Gallagher) 67259 .RE 67260 .IP \[bu] 2 67261 Onedrive 67262 .RS 2 67263 .IP \[bu] 2 67264 Implement \f[C]--poll-interval\f[R] for onedrive (Hugo Laloge) 67265 .IP \[bu] 2 67266 Add access scopes option (Sven Gerber) 67267 .RE 67268 .IP \[bu] 2 67269 Opendrive 67270 .RS 2 67271 .IP \[bu] 2 67272 Resolve lag and truncate bugs (Scott Grimes) 67273 .RE 67274 .IP \[bu] 2 67275 Pcloud 67276 .RS 2 67277 .IP \[bu] 2 67278 Fix about with no free space left (buengese) 67279 .IP \[bu] 2 67280 Fix cleanup (buengese) 67281 .RE 67282 .IP \[bu] 2 67283 S3 67284 .RS 2 67285 .IP \[bu] 2 67286 Use PUT Object instead of presigned URLs to upload single part objects 67287 (Nick Craig-Wood) 67288 .IP \[bu] 2 67289 Backend restore command to skip non-GLACIER objects (Vincent Murphy) 67290 .IP \[bu] 2 67291 Use chunksize lib to determine chunksize dynamically (Derek Battams) 67292 .IP \[bu] 2 67293 Retry RequestTimeout errors (Nick Craig-Wood) 67294 .IP \[bu] 2 67295 Implement reading and writing of metadata (Nick Craig-Wood) 67296 .RE 67297 .IP \[bu] 2 67298 SFTP 67299 .RS 2 67300 .IP \[bu] 2 67301 Add support for about and hashsum on windows server (albertony) 67302 .IP \[bu] 2 67303 Use vendor-specific VFS statistics extension for about if available 67304 (albertony) 67305 .IP \[bu] 2 67306 Add \f[C]--sftp-chunk-size\f[R] to control packets sizes for high 67307 latency links (Nick Craig-Wood) 67308 .IP \[bu] 2 67309 Add \f[C]--sftp-concurrency\f[R] to improve high latency transfers (Nick 67310 Craig-Wood) 67311 .IP \[bu] 2 67312 Add \f[C]--sftp-set-env\f[R] option to set environment variables (Nick 67313 Craig-Wood) 67314 .IP \[bu] 2 67315 Add Hetzner Storage Boxes to supported sftp backends (Anthrazz) 67316 .RE 67317 .IP \[bu] 2 67318 Storj 67319 .RS 2 67320 .IP \[bu] 2 67321 Fix put which lead to the file being unreadable when using mount (Erik 67322 van Velzen) 67323 .RE 67324 .IP \[bu] 2 67325 Union 67326 .RS 2 67327 .IP \[bu] 2 67328 Add \f[C]min_free_space\f[R] option for \f[C]lfs\f[R]/\f[C]eplfs\f[R] 67329 policies (Nick Craig-Wood) 67330 .IP \[bu] 2 67331 Fix uploading files to union of all bucket based remotes (Nick 67332 Craig-Wood) 67333 .IP \[bu] 2 67334 Fix get free space for remotes which don\[aq]t support it (Nick 67335 Craig-Wood) 67336 .IP \[bu] 2 67337 Fix \f[C]eplus\f[R] policy to select correct entry for existing files 67338 (Nick Craig-Wood) 67339 .IP \[bu] 2 67340 Support metadata (Nick Craig-Wood) 67341 .RE 67342 .IP \[bu] 2 67343 Uptobox 67344 .RS 2 67345 .IP \[bu] 2 67346 Fix root path handling (buengese) 67347 .RE 67348 .IP \[bu] 2 67349 WebDAV 67350 .RS 2 67351 .IP \[bu] 2 67352 Add SharePoint in other specific regions support (Noah Hsu) 67353 .RE 67354 .IP \[bu] 2 67355 Yandex 67356 .RS 2 67357 .IP \[bu] 2 67358 Handle api error on server-side move (albertony) 67359 .RE 67360 .IP \[bu] 2 67361 Zoho 67362 .RS 2 67363 .IP \[bu] 2 67364 Add Japan and China regions (buengese) 67365 .RE 67366 .SS v1.58.1 - 2022-04-29 67367 .PP 67368 See commits (https://github.com/artpar/artpar/compare/v1.58.0...v1.58.1) 67369 .IP \[bu] 2 67370 Bug Fixes 67371 .RS 2 67372 .IP \[bu] 2 67373 build: Update github.com/billziss-gh to github.com/winfsp (Nick 67374 Craig-Wood) 67375 .IP \[bu] 2 67376 filter: Fix timezone of \f[C]--min-age\f[R]/\f[C]-max-age\f[R] from UTC 67377 to local as documented (Nick Craig-Wood) 67378 .IP \[bu] 2 67379 rc/js: Correct RC method names (S\[u01A1]n Tr\[u1EA7]n-Nguy\[u1EC5]n) 67380 .IP \[bu] 2 67381 docs 67382 .RS 2 67383 .IP \[bu] 2 67384 Fix some links to command pages (albertony) 67385 .IP \[bu] 2 67386 Add \f[C]--multi-thread-streams\f[R] note to \f[C]--transfers\f[R]. 67387 (Zsolt Ero) 67388 .RE 67389 .RE 67390 .IP \[bu] 2 67391 Mount 67392 .RS 2 67393 .IP \[bu] 2 67394 Fix \f[C]--devname\f[R] and fusermount: unknown option \[aq]fsname\[aq] 67395 when mounting via rc (Nick Craig-Wood) 67396 .RE 67397 .IP \[bu] 2 67398 VFS 67399 .RS 2 67400 .IP \[bu] 2 67401 Remove wording which suggests VFS is only for mounting (Nick Craig-Wood) 67402 .RE 67403 .IP \[bu] 2 67404 Dropbox 67405 .RS 2 67406 .IP \[bu] 2 67407 Fix retries of multipart uploads with incorrect_offset error (Nick 67408 Craig-Wood) 67409 .RE 67410 .IP \[bu] 2 67411 Google Cloud Storage 67412 .RS 2 67413 .IP \[bu] 2 67414 Use the s3 pacer to speed up transactions (Nick Craig-Wood) 67415 .IP \[bu] 2 67416 pacer: Default the Google pacer to a burst of 100 to fix gcs pacing 67417 (Nick Craig-Wood) 67418 .RE 67419 .IP \[bu] 2 67420 Jottacloud 67421 .RS 2 67422 .IP \[bu] 2 67423 Fix scope in token request (albertony) 67424 .RE 67425 .IP \[bu] 2 67426 Netstorage 67427 .RS 2 67428 .IP \[bu] 2 67429 Fix unescaped HTML in documentation (Nick Craig-Wood) 67430 .IP \[bu] 2 67431 Make levels of headings consistent (Nick Craig-Wood) 67432 .IP \[bu] 2 67433 Add support contacts to netstorage doc (Nil Alexandrov) 67434 .RE 67435 .IP \[bu] 2 67436 Onedrive 67437 .RS 2 67438 .IP \[bu] 2 67439 Note that sharepoint also changes web files (.html, .aspx) (GH) 67440 .RE 67441 .IP \[bu] 2 67442 Putio 67443 .RS 2 67444 .IP \[bu] 2 67445 Handle rate limit errors (Berkan Teber) 67446 .IP \[bu] 2 67447 Fix multithread download and other ranged requests (rafma0) 67448 .RE 67449 .IP \[bu] 2 67450 S3 67451 .RS 2 67452 .IP \[bu] 2 67453 Add ChinaMobile EOS to provider list (GuoXingbin) 67454 .IP \[bu] 2 67455 Sync providers in config description with providers (Nick Craig-Wood) 67456 .RE 67457 .IP \[bu] 2 67458 SFTP 67459 .RS 2 67460 .IP \[bu] 2 67461 Fix OpenSSH 8.8+ RSA keys incompatibility (KARBOWSKI Piotr) 67462 .IP \[bu] 2 67463 Note that Scaleway C14 is deprecating SFTP in favor of S3 (Adrien 67464 Rey-Jarthon) 67465 .RE 67466 .IP \[bu] 2 67467 Storj 67468 .RS 2 67469 .IP \[bu] 2 67470 Fix bucket creation on Move (Nick Craig-Wood) 67471 .RE 67472 .IP \[bu] 2 67473 WebDAV 67474 .RS 2 67475 .IP \[bu] 2 67476 Don\[aq]t override Referer if user sets it (Nick Craig-Wood) 67477 .RE 67478 .SS v1.58.0 - 2022-03-18 67479 .PP 67480 See commits (https://github.com/artpar/artpar/compare/v1.57.0...v1.58.0) 67481 .IP \[bu] 2 67482 New backends 67483 .RS 2 67484 .IP \[bu] 2 67485 Akamai Netstorage (Nil Alexandrov) 67486 .IP \[bu] 2 67487 Seagate Lyve (https://rclone.org/s3/#lyve), SeaweedFS, Storj, RackCorp 67488 via s3 backend 67489 .IP \[bu] 2 67490 Storj (https://rclone.org/storj/) (renamed from Tardigrade - your old 67491 config files will continue working) 67492 .RE 67493 .IP \[bu] 2 67494 New commands 67495 .RS 2 67496 .IP \[bu] 2 67497 bisync (https://rclone.org/bisync/) - experimental bidirectional cloud 67498 sync (Ivan Andreev, Chris Nelson) 67499 .RE 67500 .IP \[bu] 2 67501 New Features 67502 .RS 2 67503 .IP \[bu] 2 67504 build 67505 .RS 2 67506 .IP \[bu] 2 67507 Add \f[C]windows/arm64\f[R] build (\f[C]rclone mount\f[R] not supported 67508 yet) (Nick Craig-Wood) 67509 .IP \[bu] 2 67510 Raise minimum go version to go1.15 (Nick Craig-Wood) 67511 .RE 67512 .IP \[bu] 2 67513 config: Allow dot in remote names and improve config editing (albertony) 67514 .IP \[bu] 2 67515 dedupe: Add quit as a choice in interactive mode (albertony) 67516 .IP \[bu] 2 67517 dlna: Change icons to the newest ones. 67518 (Alain Nussbaumer) 67519 .IP \[bu] 2 67520 filter: Add \f[C]{{ regexp }}\f[R] 67521 syntax (https://rclone.org/filtering/#regexp) to pattern matches (Nick 67522 Craig-Wood) 67523 .IP \[bu] 2 67524 fshttp: Add prometheus metrics for HTTP status code (Micha\[/l] Matczuk) 67525 .IP \[bu] 2 67526 hashsum: Support creating hash from data received on stdin (albertony) 67527 .IP \[bu] 2 67528 librclone 67529 .RS 2 67530 .IP \[bu] 2 67531 Allow empty string or null input instead of empty json object 67532 (albertony) 67533 .IP \[bu] 2 67534 Add support for mount commands (albertony) 67535 .RE 67536 .IP \[bu] 2 67537 operations: Add server-side moves to stats (Ole Frost) 67538 .IP \[bu] 2 67539 rc: Allow user to disable authentication for web gui (negative0) 67540 .IP \[bu] 2 67541 tree: Remove obsolete \f[C]--human\f[R] replaced by global 67542 \f[C]--human-readable\f[R] (albertony) 67543 .IP \[bu] 2 67544 version: Report correct friendly-name for newer Windows 10/11 versions 67545 (albertony) 67546 .RE 67547 .IP \[bu] 2 67548 Bug Fixes 67549 .RS 2 67550 .IP \[bu] 2 67551 build 67552 .RS 2 67553 .IP \[bu] 2 67554 Fix ARM architecture version in .deb packages after nfpm change (Nick 67555 Craig-Wood) 67556 .IP \[bu] 2 67557 Hard fork \f[C]github.com/jlaffaye/ftp\f[R] to fix 67558 \f[C]go get github.com/artpar/artpar\f[R] (Nick Craig-Wood) 67559 .RE 67560 .IP \[bu] 2 67561 oauthutil: Fix crash when webbrowser requests \f[C]/robots.txt\f[R] 67562 (Nick Craig-Wood) 67563 .IP \[bu] 2 67564 operations: Fix goroutine leak in case of copy retry (Ankur Gupta) 67565 .IP \[bu] 2 67566 rc: 67567 .RS 2 67568 .IP \[bu] 2 67569 Fix \f[C]operations/publiclink\f[R] default for \f[C]expires\f[R] 67570 parameter (Nick Craig-Wood) 67571 .IP \[bu] 2 67572 Fix missing computation of \f[C]transferQueueSize\f[R] when summing up 67573 statistics group (Carlo Mion) 67574 .IP \[bu] 2 67575 Fix missing \f[C]StatsInfo\f[R] fields in the computation of the group 67576 sum (Carlo Mion) 67577 .RE 67578 .IP \[bu] 2 67579 sync: Fix \f[C]--max-duration\f[R] so it doesn\[aq]t retry when the 67580 duration is exceeded (Nick Craig-Wood) 67581 .IP \[bu] 2 67582 touch: Fix issue where a directory is created instead of a file 67583 (albertony) 67584 .RE 67585 .IP \[bu] 2 67586 Mount 67587 .RS 2 67588 .IP \[bu] 2 67589 Add \f[C]--devname\f[R] to set the device name sent to FUSE for mount 67590 display (Nick Craig-Wood) 67591 .RE 67592 .IP \[bu] 2 67593 VFS 67594 .RS 2 67595 .IP \[bu] 2 67596 Add \f[C]vfs/stats\f[R] remote control to show statistics (Nick 67597 Craig-Wood) 67598 .IP \[bu] 2 67599 Fix 67600 \f[C]failed to _ensure cache internal error: downloaders is nil error\f[R] 67601 (Nick Craig-Wood) 67602 .IP \[bu] 2 67603 Fix handling of special characters in file names (Bumsu Hyeon) 67604 .RE 67605 .IP \[bu] 2 67606 Local 67607 .RS 2 67608 .IP \[bu] 2 67609 Fix hash invalidation which caused errors with local crypt mount (Nick 67610 Craig-Wood) 67611 .RE 67612 .IP \[bu] 2 67613 Crypt 67614 .RS 2 67615 .IP \[bu] 2 67616 Add \f[C]base64\f[R] and \f[C]base32768\f[R] filename encoding options 67617 (Max Sum, Sinan Tan) 67618 .RE 67619 .IP \[bu] 2 67620 Azure Blob 67621 .RS 2 67622 .IP \[bu] 2 67623 Implement \f[C]--azureblob-upload-concurrency\f[R] parameter to speed 67624 uploads (Nick Craig-Wood) 67625 .IP \[bu] 2 67626 Remove 100MB upper limit on \f[C]chunk_size\f[R] as it is no longer 67627 needed (Nick Craig-Wood) 67628 .IP \[bu] 2 67629 Raise \f[C]--azureblob-upload-concurrency\f[R] to 16 by default (Nick 67630 Craig-Wood) 67631 .IP \[bu] 2 67632 Fix crash with SAS URL and no container (Nick Craig-Wood) 67633 .RE 67634 .IP \[bu] 2 67635 Compress 67636 .RS 2 67637 .IP \[bu] 2 67638 Fix crash if metadata upload failed (Nick Craig-Wood) 67639 .IP \[bu] 2 67640 Fix memory leak (Nick Craig-Wood) 67641 .RE 67642 .IP \[bu] 2 67643 Drive 67644 .RS 2 67645 .IP \[bu] 2 67646 Added \f[C]--drive-copy-shortcut-content\f[R] (Abhiraj) 67647 .IP \[bu] 2 67648 Disable OAuth OOB flow (copy a token) due to Google deprecation (Nick 67649 Craig-Wood) 67650 .RS 2 67651 .IP \[bu] 2 67652 See the deprecation 67653 note (https://developers.googleblog.com/2022/02/making-oauth-flows-safer.html#disallowed-oob). 67654 .RE 67655 .IP \[bu] 2 67656 Add \f[C]--drive-skip-dangling-shortcuts\f[R] flag (Nick Craig-Wood) 67657 .IP \[bu] 2 67658 When using a link type \f[C]--drive-export-formats\f[R] shows all doc 67659 types (Nick Craig-Wood) 67660 .RE 67661 .IP \[bu] 2 67662 Dropbox 67663 .RS 2 67664 .IP \[bu] 2 67665 Speed up directory listings by specifying 1000 items in a chunk (Nick 67666 Craig-Wood) 67667 .IP \[bu] 2 67668 Save an API request when at the root (Nick Craig-Wood) 67669 .RE 67670 .IP \[bu] 2 67671 Fichier 67672 .RS 2 67673 .IP \[bu] 2 67674 Implemented About functionality (Gourav T) 67675 .RE 67676 .IP \[bu] 2 67677 FTP 67678 .RS 2 67679 .IP \[bu] 2 67680 Add \f[C]--ftp-ask-password\f[R] to prompt for password when needed 67681 (Borna Butkovic) 67682 .RE 67683 .IP \[bu] 2 67684 Google Cloud Storage 67685 .RS 2 67686 .IP \[bu] 2 67687 Add missing regions (Nick Craig-Wood) 67688 .IP \[bu] 2 67689 Disable OAuth OOB flow (copy a token) due to Google deprecation (Nick 67690 Craig-Wood) 67691 .RS 2 67692 .IP \[bu] 2 67693 See the deprecation 67694 note (https://developers.googleblog.com/2022/02/making-oauth-flows-safer.html#disallowed-oob). 67695 .RE 67696 .RE 67697 .IP \[bu] 2 67698 Googlephotos 67699 .RS 2 67700 .IP \[bu] 2 67701 Disable OAuth OOB flow (copy a token) due to Google deprecation (Nick 67702 Craig-Wood) 67703 .RS 2 67704 .IP \[bu] 2 67705 See the deprecation 67706 note (https://developers.googleblog.com/2022/02/making-oauth-flows-safer.html#disallowed-oob). 67707 .RE 67708 .RE 67709 .IP \[bu] 2 67710 Hasher 67711 .RS 2 67712 .IP \[bu] 2 67713 Fix crash on object not found (Nick Craig-Wood) 67714 .RE 67715 .IP \[bu] 2 67716 Hdfs 67717 .RS 2 67718 .IP \[bu] 2 67719 Add file (Move) and directory move (DirMove) support (Andy Jackson) 67720 .RE 67721 .IP \[bu] 2 67722 HTTP 67723 .RS 2 67724 .IP \[bu] 2 67725 Improved recognition of URL pointing to a single file (albertony) 67726 .RE 67727 .IP \[bu] 2 67728 Jottacloud 67729 .RS 2 67730 .IP \[bu] 2 67731 Change API used by recursive list (ListR) (Kim) 67732 .IP \[bu] 2 67733 Add support for Tele2 Cloud (Fredric Arklid) 67734 .RE 67735 .IP \[bu] 2 67736 Koofr 67737 .RS 2 67738 .IP \[bu] 2 67739 Add Digistorage service as a Koofr provider. 67740 (jaKa) 67741 .RE 67742 .IP \[bu] 2 67743 Mailru 67744 .RS 2 67745 .IP \[bu] 2 67746 Fix int32 overflow on arm32 (Ivan Andreev) 67747 .RE 67748 .IP \[bu] 2 67749 Onedrive 67750 .RS 2 67751 .IP \[bu] 2 67752 Add config option for oauth scope \f[C]Sites.Read.All\f[R] (Charlie 67753 Jiang) 67754 .IP \[bu] 2 67755 Minor optimization of quickxorhash (Isaac Levy) 67756 .IP \[bu] 2 67757 Add \f[C]--onedrive-root-folder-id\f[R] flag (Nick Craig-Wood) 67758 .IP \[bu] 2 67759 Do not retry on \f[C]400 pathIsTooLong\f[R] error (ctrl-q) 67760 .RE 67761 .IP \[bu] 2 67762 Pcloud 67763 .RS 2 67764 .IP \[bu] 2 67765 Add support for recursive list (ListR) (Niels van de Weem) 67766 .IP \[bu] 2 67767 Fix pre-1970 time stamps (Nick Craig-Wood) 67768 .RE 67769 .IP \[bu] 2 67770 S3 67771 .RS 2 67772 .IP \[bu] 2 67773 Use \f[C]ListObjectsV2\f[R] for faster listings (Felix Bu\[u0308]nemann) 67774 .RS 2 67775 .IP \[bu] 2 67776 Fallback to \f[C]ListObject\f[R] v1 on unsupported providers (Nick 67777 Craig-Wood) 67778 .RE 67779 .IP \[bu] 2 67780 Use the \f[C]ETag\f[R] on multipart transfers to verify the transfer was 67781 OK (Nick Craig-Wood) 67782 .RS 2 67783 .IP \[bu] 2 67784 Add \f[C]--s3-use-multipart-etag\f[R] provider quirk to disable this on 67785 unsupported providers (Nick Craig-Wood) 67786 .RE 67787 .IP \[bu] 2 67788 New Providers 67789 .RS 2 67790 .IP \[bu] 2 67791 RackCorp object storage (bbabich) 67792 .IP \[bu] 2 67793 Seagate Lyve Cloud storage (Nick Craig-Wood) 67794 .IP \[bu] 2 67795 SeaweedFS (Chris Lu) 67796 .IP \[bu] 2 67797 Storj Shared gateways (M\['a]rton Elek, Nick Craig-Wood) 67798 .RE 67799 .IP \[bu] 2 67800 Add Wasabi AP Northeast 2 endpoint info (lindwurm) 67801 .IP \[bu] 2 67802 Add \f[C]GLACIER_IR\f[R] storage class (Yunhai Luo) 67803 .IP \[bu] 2 67804 Document \f[C]Content-MD5\f[R] workaround for object-lock enabled 67805 buckets (Paulo Martins) 67806 .IP \[bu] 2 67807 Fix multipart upload with \f[C]--no-head\f[R] flag (Nick Craig-Wood) 67808 .IP \[bu] 2 67809 Simplify content length processing in s3 with download url (Logeshwaran 67810 Murugesan) 67811 .RE 67812 .IP \[bu] 2 67813 SFTP 67814 .RS 2 67815 .IP \[bu] 2 67816 Add rclone to list of supported \f[C]md5sum\f[R]/\f[C]sha1sum\f[R] 67817 commands to look for (albertony) 67818 .IP \[bu] 2 67819 Refactor so we only have one way of running remote commands (Nick 67820 Craig-Wood) 67821 .IP \[bu] 2 67822 Fix timeout on hashing large files by sending keepalives (Nick 67823 Craig-Wood) 67824 .IP \[bu] 2 67825 Fix unnecessary seeking when uploading and downloading files (Nick 67826 Craig-Wood) 67827 .IP \[bu] 2 67828 Update docs on how to create \f[C]known_hosts\f[R] file (Nick 67829 Craig-Wood) 67830 .RE 67831 .IP \[bu] 2 67832 Storj 67833 .RS 2 67834 .IP \[bu] 2 67835 Rename tardigrade backend to storj backend (Nick Craig-Wood) 67836 .IP \[bu] 2 67837 Implement server side Move for files (Nick Craig-Wood) 67838 .IP \[bu] 2 67839 Update docs to explain differences between s3 and this backend (Elek, 67840 M\['a]rton) 67841 .RE 67842 .IP \[bu] 2 67843 Swift 67844 .RS 2 67845 .IP \[bu] 2 67846 Fix About so it shows info about the current container only (Nick 67847 Craig-Wood) 67848 .RE 67849 .IP \[bu] 2 67850 Union 67851 .RS 2 67852 .IP \[bu] 2 67853 Fix treatment of remotes with \f[C]//\f[R] in (Nick Craig-Wood) 67854 .IP \[bu] 2 67855 Fix deadlock when one part of a multi-upload fails (Nick Craig-Wood) 67856 .IP \[bu] 2 67857 Fix eplus policy returned nil (Vitor Arruda) 67858 .RE 67859 .IP \[bu] 2 67860 Yandex 67861 .RS 2 67862 .IP \[bu] 2 67863 Add permanent deletion support (deinferno) 67864 .RE 67865 .SS v1.57.0 - 2021-11-01 67866 .PP 67867 See commits (https://github.com/artpar/artpar/compare/v1.56.0...v1.57.0) 67868 .IP \[bu] 2 67869 New backends 67870 .RS 2 67871 .IP \[bu] 2 67872 Sia: for Sia decentralized cloud (Ian Levesque, Matthew Sevey, Ivan 67873 Andreev) 67874 .IP \[bu] 2 67875 Hasher: caches hashes and enable hashes for backends that don\[aq]t 67876 support them (Ivan Andreev) 67877 .RE 67878 .IP \[bu] 2 67879 New commands 67880 .RS 2 67881 .IP \[bu] 2 67882 lsjson --stat: to get info about a single file/dir and 67883 \f[C]operations/stat\f[R] api (Nick Craig-Wood) 67884 .IP \[bu] 2 67885 config paths: show configured paths (albertony) 67886 .RE 67887 .IP \[bu] 2 67888 New Features 67889 .RS 2 67890 .IP \[bu] 2 67891 about: Make human-readable output more consistent with other commands 67892 (albertony) 67893 .IP \[bu] 2 67894 build 67895 .RS 2 67896 .IP \[bu] 2 67897 Use go1.17 for building and make go1.14 the minimum supported (Nick 67898 Craig-Wood) 67899 .IP \[bu] 2 67900 Update Go to 1.16 and NDK to 22b for Android builds (x0b) 67901 .RE 67902 .IP \[bu] 2 67903 config 67904 .RS 2 67905 .IP \[bu] 2 67906 Support hyphen in remote name from environment variable (albertony) 67907 .IP \[bu] 2 67908 Make temporary directory user-configurable (albertony) 67909 .IP \[bu] 2 67910 Convert \f[C]--cache-dir\f[R] value to an absolute path (albertony) 67911 .IP \[bu] 2 67912 Do not override MIME types from OS defaults (albertony) 67913 .RE 67914 .IP \[bu] 2 67915 docs 67916 .RS 2 67917 .IP \[bu] 2 67918 Toc styling and header levels cleanup (albertony) 67919 .IP \[bu] 2 67920 Extend documentation on valid remote names (albertony) 67921 .IP \[bu] 2 67922 Mention make for building and cmount tag for macos (Alex Chen) 67923 .IP \[bu] 2 67924 \&...and many more contributions to numerous to mention! 67925 .RE 67926 .IP \[bu] 2 67927 fs: Move with \f[C]--ignore-existing\f[R] will not delete skipped files 67928 (Nathan Collins) 67929 .IP \[bu] 2 67930 hashsum 67931 .RS 2 67932 .IP \[bu] 2 67933 Treat hash values in sum file as case insensitive (Ivan Andreev) 67934 .IP \[bu] 2 67935 Don\[aq]t put \f[C]ERROR\f[R] or \f[C]UNSUPPORTED\f[R] in output (Ivan 67936 Andreev) 67937 .RE 67938 .IP \[bu] 2 67939 lib/encoder: Add encoding of square brackets (Ivan Andreev) 67940 .IP \[bu] 2 67941 lib/file: Improve error message when attempting to create dir on 67942 nonexistent drive on windows (albertony) 67943 .IP \[bu] 2 67944 lib/http: Factor password hash salt into options with default (Nolan 67945 Woods) 67946 .IP \[bu] 2 67947 lib/kv: Add key-value database api (Ivan Andreev) 67948 .IP \[bu] 2 67949 librclone 67950 .RS 2 67951 .IP \[bu] 2 67952 Add \f[C]RcloneFreeString\f[R] function (albertony) 67953 .IP \[bu] 2 67954 Free strings in python example (albertony) 67955 .RE 67956 .IP \[bu] 2 67957 log: Optionally print pid in logs (Ivan Andreev) 67958 .IP \[bu] 2 67959 ls: Introduce \f[C]--human-readable\f[R] global option to print 67960 human-readable sizes (albertony) 67961 .IP \[bu] 2 67962 ncdu: Introduce key \f[C]u\f[R] to toggle human-readable (albertony) 67963 .IP \[bu] 2 67964 operations: Add \f[C]rmdirs -v\f[R] output (Justin Winokur) 67965 .IP \[bu] 2 67966 serve sftp 67967 .RS 2 67968 .IP \[bu] 2 67969 Generate an ECDSA server key as well as RSA (Nick Craig-Wood) 67970 .IP \[bu] 2 67971 Generate an Ed25519 server key as well as ECDSA and RSA (albertony) 67972 .RE 67973 .IP \[bu] 2 67974 serve docker 67975 .RS 2 67976 .IP \[bu] 2 67977 Allow to customize proxy settings of docker plugin (Ivan Andreev) 67978 .IP \[bu] 2 67979 Build docker plugin for multiple platforms (Thomas Stachl) 67980 .RE 67981 .IP \[bu] 2 67982 size: Include human-readable count (albertony) 67983 .IP \[bu] 2 67984 touch: Add support for touching files in directory, with recursive 67985 option, filtering and \f[C]--dry-run\f[R]/\f[C]-i\f[R] (albertony) 67986 .IP \[bu] 2 67987 tree: Option to print human-readable sizes removed in favor of global 67988 option (albertony) 67989 .RE 67990 .IP \[bu] 2 67991 Bug Fixes 67992 .RS 2 67993 .IP \[bu] 2 67994 lib/http 67995 .RS 2 67996 .IP \[bu] 2 67997 Fix bad username check in single auth secret provider (Nolan Woods) 67998 .IP \[bu] 2 67999 Fix handling of SSL credentials (Nolan Woods) 68000 .RE 68001 .IP \[bu] 2 68002 serve ftp: Ensure modtime is passed as UTC always to fix timezone 68003 oddities (Nick Craig-Wood) 68004 .IP \[bu] 2 68005 serve sftp: Fix generation of server keys on windows (albertony) 68006 .IP \[bu] 2 68007 serve docker: Fix octal umask (Ivan Andreev) 68008 .RE 68009 .IP \[bu] 2 68010 Mount 68011 .RS 2 68012 .IP \[bu] 2 68013 Enable rclone to be run as mount helper direct from the fstab (Ivan 68014 Andreev) 68015 .IP \[bu] 2 68016 Use procfs to validate mount on linux (Ivan Andreev) 68017 .IP \[bu] 2 68018 Correctly daemonize for compatibility with automount (Ivan Andreev) 68019 .RE 68020 .IP \[bu] 2 68021 VFS 68022 .RS 2 68023 .IP \[bu] 2 68024 Ensure names used in cache path are legal on current OS (albertony) 68025 .IP \[bu] 2 68026 Ignore \f[C]ECLOSED\f[R] when truncating file handles to fix 68027 intermittent bad file descriptor error (Nick Craig-Wood) 68028 .RE 68029 .IP \[bu] 2 68030 Local 68031 .RS 2 68032 .IP \[bu] 2 68033 Refactor default OS encoding out from local backend into shared encoder 68034 lib (albertony) 68035 .RE 68036 .IP \[bu] 2 68037 Crypt 68038 .RS 2 68039 .IP \[bu] 2 68040 Return wrapped object even with \f[C]--crypt-no-data-encryption\f[R] 68041 (Ivan Andreev) 68042 .IP \[bu] 2 68043 Fix uploads with \f[C]--crypt-no-data-encryption\f[R] (Nick Craig-Wood) 68044 .RE 68045 .IP \[bu] 2 68046 Azure Blob 68047 .RS 2 68048 .IP \[bu] 2 68049 Add \f[C]--azureblob-no-head-object\f[R] (Tatsuya Noyori) 68050 .RE 68051 .IP \[bu] 2 68052 Box 68053 .RS 2 68054 .IP \[bu] 2 68055 Make listings of heavily used directories more reliable (Nick 68056 Craig-Wood) 68057 .IP \[bu] 2 68058 When doing cleanup delete as much as possible (Nick Craig-Wood) 68059 .IP \[bu] 2 68060 Add \f[C]--box-list-chunk\f[R] to control listing chunk size (Nick 68061 Craig-Wood) 68062 .IP \[bu] 2 68063 Delete items in parallel in cleanup using \f[C]--checkers\f[R] threads 68064 (Nick Craig-Wood) 68065 .IP \[bu] 2 68066 Add \f[C]--box-owned-by\f[R] to only show items owned by the login 68067 passed (Nick Craig-Wood) 68068 .IP \[bu] 2 68069 Retry \f[C]operation_blocked_temporary\f[R] errors (Nick Craig-Wood) 68070 .RE 68071 .IP \[bu] 2 68072 Chunker 68073 .RS 2 68074 .IP \[bu] 2 68075 Md5all must create metadata if base hash is slow (Ivan Andreev) 68076 .RE 68077 .IP \[bu] 2 68078 Drive 68079 .RS 2 68080 .IP \[bu] 2 68081 Speed up directory listings by constraining the API listing using the 68082 current filters (fotile96, Ivan Andreev) 68083 .IP \[bu] 2 68084 Fix buffering for single request upload for files smaller than 68085 \f[C]--drive-upload-cutoff\f[R] (YenForYang) 68086 .IP \[bu] 2 68087 Add \f[C]-o config\f[R] option to \f[C]backend drives\f[R] to make 68088 config for all shared drives (Nick Craig-Wood) 68089 .RE 68090 .IP \[bu] 2 68091 Dropbox 68092 .RS 2 68093 .IP \[bu] 2 68094 Add \f[C]--dropbox-batch-commit-timeout\f[R] to control batch timeout 68095 (Nick Craig-Wood) 68096 .RE 68097 .IP \[bu] 2 68098 Filefabric 68099 .RS 2 68100 .IP \[bu] 2 68101 Make backoff exponential for error_background to fix errors (Nick 68102 Craig-Wood) 68103 .IP \[bu] 2 68104 Fix directory move after API change (Nick Craig-Wood) 68105 .RE 68106 .IP \[bu] 2 68107 FTP 68108 .RS 2 68109 .IP \[bu] 2 68110 Enable tls session cache by default (Ivan Andreev) 68111 .IP \[bu] 2 68112 Add option to disable tls13 (Ivan Andreev) 68113 .IP \[bu] 2 68114 Fix timeout after long uploads (Ivan Andreev) 68115 .IP \[bu] 2 68116 Add support for precise time (Ivan Andreev) 68117 .IP \[bu] 2 68118 Enable CI for ProFtpd, PureFtpd, VsFtpd (Ivan Andreev) 68119 .RE 68120 .IP \[bu] 2 68121 Googlephotos 68122 .RS 2 68123 .IP \[bu] 2 68124 Use encoder for album names to fix albums with control characters (Parth 68125 Shukla) 68126 .RE 68127 .IP \[bu] 2 68128 Jottacloud 68129 .RS 2 68130 .IP \[bu] 2 68131 Implement \f[C]SetModTime\f[R] to support modtime-only changes 68132 (albertony) 68133 .IP \[bu] 2 68134 Improved error handling with \f[C]SetModTime\f[R] and corrupt files in 68135 general (albertony) 68136 .IP \[bu] 2 68137 Add support for \f[C]UserInfo\f[R] (\f[C]rclone config userinfo\f[R]) 68138 feature (albertony) 68139 .IP \[bu] 2 68140 Return direct download link from \f[C]rclone link\f[R] command 68141 (albertony) 68142 .RE 68143 .IP \[bu] 2 68144 Koofr 68145 .RS 2 68146 .IP \[bu] 2 68147 Create direct share link (Dmitry Bogatov) 68148 .RE 68149 .IP \[bu] 2 68150 Pcloud 68151 .RS 2 68152 .IP \[bu] 2 68153 Add sha256 support (Ken Enrique Morel) 68154 .RE 68155 .IP \[bu] 2 68156 Premiumizeme 68157 .RS 2 68158 .IP \[bu] 2 68159 Fix directory listing after API changes (Nick Craig-Wood) 68160 .IP \[bu] 2 68161 Fix server side move after API change (Nick Craig-Wood) 68162 .IP \[bu] 2 68163 Fix server side directory move after API changes (Nick Craig-Wood) 68164 .RE 68165 .IP \[bu] 2 68166 S3 68167 .RS 2 68168 .IP \[bu] 2 68169 Add support to use CDN URL to download the file (Logeshwaran) 68170 .IP \[bu] 2 68171 Add AWS Snowball Edge to providers examples (r0kk3rz) 68172 .IP \[bu] 2 68173 Use a combination of SDK retries and rclone retries (Nick Craig-Wood) 68174 .IP \[bu] 2 68175 Fix IAM Role for Service Account not working and other auth problems 68176 (Nick Craig-Wood) 68177 .IP \[bu] 2 68178 Fix \f[C]shared_credentials_file\f[R] auth after reverting incorrect fix 68179 (Nick Craig-Wood) 68180 .IP \[bu] 2 68181 Fix corrupted on transfer: sizes differ 0 vs xxxx with Ceph (Nick 68182 Craig-Wood) 68183 .RE 68184 .IP \[bu] 2 68185 Seafile 68186 .RS 2 68187 .IP \[bu] 2 68188 Fix error when not configured for 2fa (Fred) 68189 .RE 68190 .IP \[bu] 2 68191 SFTP 68192 .RS 2 68193 .IP \[bu] 2 68194 Fix timeout when doing MD5SUM of large file (Nick Craig-Wood) 68195 .RE 68196 .IP \[bu] 2 68197 Swift 68198 .RS 2 68199 .IP \[bu] 2 68200 Update OCI URL (David Liu) 68201 .IP \[bu] 2 68202 Document OVH Cloud Archive (HNGamingUK) 68203 .RE 68204 .IP \[bu] 2 68205 Union 68206 .RS 2 68207 .IP \[bu] 2 68208 Fix rename not working with union of local disk and bucket based remote 68209 (Nick Craig-Wood) 68210 .RE 68211 .SS v1.56.2 - 2021-10-01 68212 .PP 68213 See commits (https://github.com/artpar/artpar/compare/v1.56.1...v1.56.2) 68214 .IP \[bu] 2 68215 Bug Fixes 68216 .RS 2 68217 .IP \[bu] 2 68218 serve http: Re-add missing auth to http service (Nolan Woods) 68219 .IP \[bu] 2 68220 build: Update golang.org/x/sys to fix crash on macOS when compiled with 68221 go1.17 (Herby Gillot) 68222 .RE 68223 .IP \[bu] 2 68224 FTP 68225 .RS 2 68226 .IP \[bu] 2 68227 Fix deadlock after failed update when concurrency=1 (Ivan Andreev) 68228 .RE 68229 .SS v1.56.1 - 2021-09-19 68230 .PP 68231 See commits (https://github.com/artpar/artpar/compare/v1.56.0...v1.56.1) 68232 .IP \[bu] 2 68233 Bug Fixes 68234 .RS 2 68235 .IP \[bu] 2 68236 accounting: Fix maximum bwlimit by scaling scale max token bucket size 68237 (Nick Craig-Wood) 68238 .IP \[bu] 2 68239 rc: Fix speed does not update in core/stats (negative0) 68240 .IP \[bu] 2 68241 selfupdate: Fix \f[C]--quiet\f[R] option, not quite quiet (yedamo) 68242 .IP \[bu] 2 68243 serve http: Fix \f[C]serve http\f[R] exiting directly after starting 68244 (Cnly) 68245 .IP \[bu] 2 68246 build 68247 .RS 2 68248 .IP \[bu] 2 68249 Apply gofmt from golang 1.17 (Ivan Andreev) 68250 .IP \[bu] 2 68251 Update Go to 1.16 and NDK to 22b for android/any (x0b) 68252 .RE 68253 .RE 68254 .IP \[bu] 2 68255 Mount 68256 .RS 2 68257 .IP \[bu] 2 68258 Fix \f[C]--daemon\f[R] mode (Ivan Andreev) 68259 .RE 68260 .IP \[bu] 2 68261 VFS 68262 .RS 2 68263 .IP \[bu] 2 68264 Fix duplicates on rename (Nick Craig-Wood) 68265 .IP \[bu] 2 68266 Fix crash when truncating a just uploaded object (Nick Craig-Wood) 68267 .IP \[bu] 2 68268 Fix issue where empty dirs would build up in cache meta dir (albertony) 68269 .RE 68270 .IP \[bu] 2 68271 Drive 68272 .RS 2 68273 .IP \[bu] 2 68274 Fix instructions for auto config (Greg Sadetsky) 68275 .IP \[bu] 2 68276 Fix lsf example without drive-impersonate (Greg Sadetsky) 68277 .RE 68278 .IP \[bu] 2 68279 Onedrive 68280 .RS 2 68281 .IP \[bu] 2 68282 Handle HTTP 400 better in PublicLink (Alex Chen) 68283 .IP \[bu] 2 68284 Clarification of the process for creating custom client_id (Mariano 68285 Absatz) 68286 .RE 68287 .IP \[bu] 2 68288 Pcloud 68289 .RS 2 68290 .IP \[bu] 2 68291 Return an early error when Put is called with an unknown size (Nick 68292 Craig-Wood) 68293 .IP \[bu] 2 68294 Try harder to delete a failed upload (Nick Craig-Wood) 68295 .RE 68296 .IP \[bu] 2 68297 S3 68298 .RS 2 68299 .IP \[bu] 2 68300 Add Wasabi\[aq]s AP-Northeast endpoint info (hota) 68301 .IP \[bu] 2 68302 Fix typo in s3 documentation (Greg Sadetsky) 68303 .RE 68304 .IP \[bu] 2 68305 Seafile 68306 .RS 2 68307 .IP \[bu] 2 68308 Fix 2fa config state machine (Fred) 68309 .RE 68310 .IP \[bu] 2 68311 SFTP 68312 .RS 2 68313 .IP \[bu] 2 68314 Remove spurious error message on 68315 \f[C]--sftp-disable-concurrent-reads\f[R] (Nick Craig-Wood) 68316 .RE 68317 .IP \[bu] 2 68318 Sugarsync 68319 .RS 2 68320 .IP \[bu] 2 68321 Fix initial connection after config re-arrangement (Nick Craig-Wood) 68322 .RE 68323 .SS v1.56.0 - 2021-07-20 68324 .PP 68325 See commits (https://github.com/artpar/artpar/compare/v1.55.0...v1.56.0) 68326 .IP \[bu] 2 68327 New backends 68328 .RS 2 68329 .IP \[bu] 2 68330 Uptobox (https://rclone.org/uptobox/) (buengese) 68331 .RE 68332 .IP \[bu] 2 68333 New commands 68334 .RS 2 68335 .IP \[bu] 2 68336 serve docker (https://rclone.org/commands/rclone_serve_docker/) (Antoine 68337 GIRARD) (Ivan Andreev) 68338 .RS 2 68339 .IP \[bu] 2 68340 and accompanying docker volume plugin (https://rclone.org/docker/) 68341 .RE 68342 .IP \[bu] 2 68343 checksum (https://rclone.org/commands/rclone_checksum/) to check files 68344 against a file of checksums (Ivan Andreev) 68345 .RS 2 68346 .IP \[bu] 2 68347 this is also available as \f[C]rclone md5sum -C\f[R] etc 68348 .RE 68349 .IP \[bu] 2 68350 config touch (https://rclone.org/commands/rclone_config_touch/): ensure 68351 config exists at configured location (albertony) 68352 .IP \[bu] 2 68353 test 68354 changenotify (https://rclone.org/commands/rclone_test_changenotify/): 68355 command to help debugging changenotify (Nick Craig-Wood) 68356 .RE 68357 .IP \[bu] 2 68358 Deprecations 68359 .RS 2 68360 .IP \[bu] 2 68361 \f[C]dbhashsum\f[R]: Remove command deprecated a year ago (Ivan Andreev) 68362 .IP \[bu] 2 68363 \f[C]cache\f[R]: Deprecate cache backend (Ivan Andreev) 68364 .RE 68365 .IP \[bu] 2 68366 New Features 68367 .RS 2 68368 .IP \[bu] 2 68369 rework config system so it can be used non-interactively via cli and rc 68370 API. 68371 .RS 2 68372 .IP \[bu] 2 68373 See docs in config 68374 create (https://rclone.org/commands/rclone_config_create/) 68375 .IP \[bu] 2 68376 This is a very big change to all the backends so may cause breakages - 68377 please file bugs! 68378 .RE 68379 .IP \[bu] 2 68380 librclone - export the rclone RC as a C library (lewisxy) (Nick 68381 Craig-Wood) 68382 .RS 2 68383 .IP \[bu] 2 68384 Link a C-API rclone shared object into your project 68385 .IP \[bu] 2 68386 Use the RC as an in memory interface 68387 .IP \[bu] 2 68388 Python example supplied 68389 .IP \[bu] 2 68390 Also supports Android and gomobile 68391 .RE 68392 .IP \[bu] 2 68393 fs 68394 .RS 2 68395 .IP \[bu] 2 68396 Add \f[C]--disable-http2\f[R] for global http2 disable (Nick Craig-Wood) 68397 .IP \[bu] 2 68398 Make \f[C]--dump\f[R] imply \f[C]-vv\f[R] (Alex Chen) 68399 .IP \[bu] 2 68400 Use binary prefixes for size and rate units (albertony) 68401 .IP \[bu] 2 68402 Use decimal prefixes for counts (albertony) 68403 .IP \[bu] 2 68404 Add google search widget to rclone.org (Ivan Andreev) 68405 .RE 68406 .IP \[bu] 2 68407 accounting: Calculate rolling average speed (Haochen Tong) 68408 .IP \[bu] 2 68409 atexit: Terminate with non-zero status after receiving signal (Michael 68410 Hanselmann) 68411 .IP \[bu] 2 68412 build 68413 .RS 2 68414 .IP \[bu] 2 68415 Only run event-based workflow scripts under rclone repo with manual 68416 override (Mathieu Carbou) 68417 .IP \[bu] 2 68418 Add Android build with gomobile (x0b) 68419 .RE 68420 .IP \[bu] 2 68421 check: Log the hash in use like cryptcheck does (Nick Craig-Wood) 68422 .IP \[bu] 2 68423 version: Print os/version, kernel and bitness (Ivan Andreev) 68424 .IP \[bu] 2 68425 config 68426 .RS 2 68427 .IP \[bu] 2 68428 Prevent use of Windows reserved names in config file name (albertony) 68429 .IP \[bu] 2 68430 Create config file in windows appdata directory by default (albertony) 68431 .IP \[bu] 2 68432 Treat any config file paths with filename notfound as memory-only config 68433 (albertony) 68434 .IP \[bu] 2 68435 Delay load config file (albertony) 68436 .IP \[bu] 2 68437 Replace defaultConfig with a thread-safe in-memory implementation (Chris 68438 Macklin) 68439 .IP \[bu] 2 68440 Allow \f[C]config create\f[R] and friends to take \f[C]key=value\f[R] 68441 parameters (Nick Craig-Wood) 68442 .IP \[bu] 2 68443 Fixed issues with flags/options set by environment vars. 68444 (Ole Frost) 68445 .RE 68446 .IP \[bu] 2 68447 fshttp: Implement graceful DSCP error handling (Tyson Moore) 68448 .IP \[bu] 2 68449 lib/http - provides an abstraction for a central http server that 68450 services can bind routes to (Nolan Woods) 68451 .RS 2 68452 .IP \[bu] 2 68453 Add \f[C]--template\f[R] config and flags to serve/data (Nolan Woods) 68454 .IP \[bu] 2 68455 Add default 404 handler (Nolan Woods) 68456 .RE 68457 .IP \[bu] 2 68458 link: Use \[dq]off\[dq] value for unset expiry (Nick Craig-Wood) 68459 .IP \[bu] 2 68460 oauthutil: Raise fatal error if token expired without refresh token 68461 (Alex Chen) 68462 .IP \[bu] 2 68463 rcat: Add \f[C]--size\f[R] flag for more efficient uploads of known size 68464 (Nazar Mishturak) 68465 .IP \[bu] 2 68466 serve sftp: Add \f[C]--stdio\f[R] flag to serve via stdio (Tom) 68467 .IP \[bu] 2 68468 sync: Don\[aq]t warn about \f[C]--no-traverse\f[R] when 68469 \f[C]--files-from\f[R] is set (Nick Gaya) 68470 .IP \[bu] 2 68471 \f[C]test makefiles\f[R] 68472 .RS 2 68473 .IP \[bu] 2 68474 Add \f[C]--seed\f[R] flag and make data generated repeatable (Nick 68475 Craig-Wood) 68476 .IP \[bu] 2 68477 Add log levels and speed summary (Nick Craig-Wood) 68478 .RE 68479 .RE 68480 .IP \[bu] 2 68481 Bug Fixes 68482 .RS 2 68483 .IP \[bu] 2 68484 accounting: Fix startTime of statsGroups.sum (Haochen Tong) 68485 .IP \[bu] 2 68486 cmd/ncdu: Fix out of range panic in delete (buengese) 68487 .IP \[bu] 2 68488 config 68489 .RS 2 68490 .IP \[bu] 2 68491 Fix issues with memory-only config file paths (albertony) 68492 .IP \[bu] 2 68493 Fix in memory config not saving on the fly backend config (Nick 68494 Craig-Wood) 68495 .RE 68496 .IP \[bu] 2 68497 fshttp: Fix address parsing for DSCP (Tyson Moore) 68498 .IP \[bu] 2 68499 ncdu: Update termbox-go library to fix crash (Nick Craig-Wood) 68500 .IP \[bu] 2 68501 oauthutil: Fix old authorize result not recognised (Cnly) 68502 .IP \[bu] 2 68503 operations: Don\[aq]t update timestamps of files in 68504 \f[C]--compare-dest\f[R] (Nick Gaya) 68505 .IP \[bu] 2 68506 selfupdate: fix archive name on macos (Ivan Andreev) 68507 .RE 68508 .IP \[bu] 2 68509 Mount 68510 .RS 2 68511 .IP \[bu] 2 68512 Refactor before adding serve docker (Antoine GIRARD) 68513 .RE 68514 .IP \[bu] 2 68515 VFS 68516 .RS 2 68517 .IP \[bu] 2 68518 Add cache reset for \f[C]--vfs-cache-max-size\f[R] handling at cache 68519 poll interval (Leo Luan) 68520 .IP \[bu] 2 68521 Fix modtime changing when reading file into cache (Nick Craig-Wood) 68522 .IP \[bu] 2 68523 Avoid unnecessary subdir in cache path (albertony) 68524 .IP \[bu] 2 68525 Fix that umask option cannot be set as environment variable (albertony) 68526 .IP \[bu] 2 68527 Do not print notice about missing poll-interval support when set to 0 68528 (albertony) 68529 .RE 68530 .IP \[bu] 2 68531 Local 68532 .RS 2 68533 .IP \[bu] 2 68534 Always use readlink to read symlink size for better compatibility (Nick 68535 Craig-Wood) 68536 .IP \[bu] 2 68537 Add \f[C]--local-unicode-normalization\f[R] (and remove 68538 \f[C]--local-no-unicode-normalization\f[R]) (Nick Craig-Wood) 68539 .IP \[bu] 2 68540 Skip entries removed concurrently with List() (Ivan Andreev) 68541 .RE 68542 .IP \[bu] 2 68543 Crypt 68544 .RS 2 68545 .IP \[bu] 2 68546 Support timestamped filenames from \f[C]--b2-versions\f[R] (Dominik 68547 Mydlil) 68548 .RE 68549 .IP \[bu] 2 68550 B2 68551 .RS 2 68552 .IP \[bu] 2 68553 Don\[aq]t include the bucket name in public link file prefixes (Jeffrey 68554 Tolar) 68555 .IP \[bu] 2 68556 Fix versions and .files with no extension (Nick Craig-Wood) 68557 .IP \[bu] 2 68558 Factor version handling into lib/version (Dominik Mydlil) 68559 .RE 68560 .IP \[bu] 2 68561 Box 68562 .RS 2 68563 .IP \[bu] 2 68564 Use upload preflight check to avoid listings in file uploads (Nick 68565 Craig-Wood) 68566 .IP \[bu] 2 68567 Return errors instead of calling log.Fatal with them (Nick Craig-Wood) 68568 .RE 68569 .IP \[bu] 2 68570 Drive 68571 .RS 2 68572 .IP \[bu] 2 68573 Switch to the Drives API for looking up shared drives (Nick Craig-Wood) 68574 .IP \[bu] 2 68575 Fix some google docs being treated as files (Nick Craig-Wood) 68576 .RE 68577 .IP \[bu] 2 68578 Dropbox 68579 .RS 2 68580 .IP \[bu] 2 68581 Add \f[C]--dropbox-batch-mode\f[R] flag to speed up uploading (Nick 68582 Craig-Wood) 68583 .RS 2 68584 .IP \[bu] 2 68585 Read the batch mode (https://rclone.org/dropbox/#batch-mode) docs for 68586 more info 68587 .RE 68588 .IP \[bu] 2 68589 Set visibility in link sharing when \f[C]--expire\f[R] is set (Nick 68590 Craig-Wood) 68591 .IP \[bu] 2 68592 Simplify chunked uploads (Alexey Ivanov) 68593 .IP \[bu] 2 68594 Improve \[dq]own App IP\[dq] instructions (Ivan Andreev) 68595 .RE 68596 .IP \[bu] 2 68597 Fichier 68598 .RS 2 68599 .IP \[bu] 2 68600 Check if more than one upload link is returned (Nick Craig-Wood) 68601 .IP \[bu] 2 68602 Support downloading password protected files and folders (Florian 68603 Penzkofer) 68604 .IP \[bu] 2 68605 Make error messages report text from the API (Nick Craig-Wood) 68606 .IP \[bu] 2 68607 Fix move of files in the same directory (Nick Craig-Wood) 68608 .IP \[bu] 2 68609 Check that we actually got a download token and retry if we didn\[aq]t 68610 (buengese) 68611 .RE 68612 .IP \[bu] 2 68613 Filefabric 68614 .RS 2 68615 .IP \[bu] 2 68616 Fix listing after change of from field from \[dq]int\[dq] to int. 68617 (Nick Craig-Wood) 68618 .RE 68619 .IP \[bu] 2 68620 FTP 68621 .RS 2 68622 .IP \[bu] 2 68623 Make upload error 250 indicate success (Nick Craig-Wood) 68624 .RE 68625 .IP \[bu] 2 68626 GCS 68627 .RS 2 68628 .IP \[bu] 2 68629 Make compatible with gsutil\[aq]s mtime metadata (database64128) 68630 .IP \[bu] 2 68631 Clean up time format constants (database64128) 68632 .RE 68633 .IP \[bu] 2 68634 Google Photos 68635 .RS 2 68636 .IP \[bu] 2 68637 Fix read only scope not being used properly (Nick Craig-Wood) 68638 .RE 68639 .IP \[bu] 2 68640 HTTP 68641 .RS 2 68642 .IP \[bu] 2 68643 Replace httplib with lib/http (Nolan Woods) 68644 .IP \[bu] 2 68645 Clean up Bind to better use middleware (Nolan Woods) 68646 .RE 68647 .IP \[bu] 2 68648 Jottacloud 68649 .RS 2 68650 .IP \[bu] 2 68651 Fix legacy auth with state based config system (buengese) 68652 .IP \[bu] 2 68653 Fix invalid url in output from link command (albertony) 68654 .IP \[bu] 2 68655 Add no versions option (buengese) 68656 .RE 68657 .IP \[bu] 2 68658 Onedrive 68659 .RS 2 68660 .IP \[bu] 2 68661 Add \f[C]list_chunk option\f[R] (Nick Gaya) 68662 .IP \[bu] 2 68663 Also report root error if unable to cancel multipart upload (Cnly) 68664 .IP \[bu] 2 68665 Fix failed to configure: empty token found error (Nick Craig-Wood) 68666 .IP \[bu] 2 68667 Make link return direct download link (Xuanchen Wu) 68668 .RE 68669 .IP \[bu] 2 68670 S3 68671 .RS 2 68672 .IP \[bu] 2 68673 Add \f[C]--s3-no-head-object\f[R] (Tatsuya Noyori) 68674 .IP \[bu] 2 68675 Remove WebIdentityRoleProvider to fix crash on auth (Nick Craig-Wood) 68676 .IP \[bu] 2 68677 Don\[aq]t check to see if remote is object if it ends with / (Nick 68678 Craig-Wood) 68679 .IP \[bu] 2 68680 Add SeaweedFS (Chris Lu) 68681 .IP \[bu] 2 68682 Update Alibaba OSS endpoints (Chuan Zh) 68683 .RE 68684 .IP \[bu] 2 68685 SFTP 68686 .RS 2 68687 .IP \[bu] 2 68688 Fix performance regression by re-enabling concurrent writes (Nick 68689 Craig-Wood) 68690 .IP \[bu] 2 68691 Expand tilde and environment variables in configured 68692 \f[C]known_hosts_file\f[R] (albertony) 68693 .RE 68694 .IP \[bu] 2 68695 Tardigrade 68696 .RS 2 68697 .IP \[bu] 2 68698 Upgrade to uplink v1.4.6 (Caleb Case) 68699 .IP \[bu] 2 68700 Use negative offset (Caleb Case) 68701 .IP \[bu] 2 68702 Add warning about \f[C]too many open files\f[R] (acsfer) 68703 .RE 68704 .IP \[bu] 2 68705 WebDAV 68706 .RS 2 68707 .IP \[bu] 2 68708 Fix sharepoint auth over http (Nick Craig-Wood) 68709 .IP \[bu] 2 68710 Add headers option (Antoon Prins) 68711 .RE 68712 .SS v1.55.1 - 2021-04-26 68713 .PP 68714 See commits (https://github.com/artpar/artpar/compare/v1.55.0...v1.55.1) 68715 .IP \[bu] 2 68716 Bug Fixes 68717 .RS 2 68718 .IP \[bu] 2 68719 selfupdate 68720 .RS 2 68721 .IP \[bu] 2 68722 Dont detect FUSE if build is static (Ivan Andreev) 68723 .IP \[bu] 2 68724 Add build tag noselfupdate (Ivan Andreev) 68725 .RE 68726 .IP \[bu] 2 68727 sync: Fix incorrect error reported by graceful cutoff (Nick Craig-Wood) 68728 .IP \[bu] 2 68729 install.sh: fix macOS arm64 download (Nick Craig-Wood) 68730 .IP \[bu] 2 68731 build: Fix version numbers in android branch builds (Nick Craig-Wood) 68732 .IP \[bu] 2 68733 docs 68734 .RS 2 68735 .IP \[bu] 2 68736 Contributing.md: update setup instructions for go1.16 (Nick Gaya) 68737 .IP \[bu] 2 68738 WinFsp 2021 is out of beta (albertony) 68739 .IP \[bu] 2 68740 Minor cleanup of space around code section (albertony) 68741 .IP \[bu] 2 68742 Fixed some typos (albertony) 68743 .RE 68744 .RE 68745 .IP \[bu] 2 68746 VFS 68747 .RS 2 68748 .IP \[bu] 2 68749 Fix a code path which allows dirty data to be removed causing data loss 68750 (Nick Craig-Wood) 68751 .RE 68752 .IP \[bu] 2 68753 Compress 68754 .RS 2 68755 .IP \[bu] 2 68756 Fix compressed name regexp (buengese) 68757 .RE 68758 .IP \[bu] 2 68759 Drive 68760 .RS 2 68761 .IP \[bu] 2 68762 Fix backend copyid of google doc to directory (Nick Craig-Wood) 68763 .IP \[bu] 2 68764 Don\[aq]t open browser when service account... 68765 (Ansh Mittal) 68766 .RE 68767 .IP \[bu] 2 68768 Dropbox 68769 .RS 2 68770 .IP \[bu] 2 68771 Add missing team_data.member scope for use with --impersonate (Nick 68772 Craig-Wood) 68773 .IP \[bu] 2 68774 Fix About after scopes changes - rclone config reconnect needed (Nick 68775 Craig-Wood) 68776 .IP \[bu] 2 68777 Fix Unable to decrypt returned paths from changeNotify (Nick Craig-Wood) 68778 .RE 68779 .IP \[bu] 2 68780 FTP 68781 .RS 2 68782 .IP \[bu] 2 68783 Fix implicit TLS (Ivan Andreev) 68784 .RE 68785 .IP \[bu] 2 68786 Onedrive 68787 .RS 2 68788 .IP \[bu] 2 68789 Work around for random \[dq]Unable to initialize RPS\[dq] errors 68790 (OleFrost) 68791 .RE 68792 .IP \[bu] 2 68793 SFTP 68794 .RS 2 68795 .IP \[bu] 2 68796 Revert sftp library to v1.12.0 from v1.13.0 to fix performance 68797 regression (Nick Craig-Wood) 68798 .IP \[bu] 2 68799 Fix Update ReadFrom failed: failed to send packet: EOF errors (Nick 68800 Craig-Wood) 68801 .RE 68802 .IP \[bu] 2 68803 Zoho 68804 .RS 2 68805 .IP \[bu] 2 68806 Fix error when region isn\[aq]t set (buengese) 68807 .IP \[bu] 2 68808 Do not ask for mountpoint twice when using headless setup (buengese) 68809 .RE 68810 .SS v1.55.0 - 2021-03-31 68811 .PP 68812 See commits (https://github.com/artpar/artpar/compare/v1.54.0...v1.55.0) 68813 .IP \[bu] 2 68814 New commands 68815 .RS 2 68816 .IP \[bu] 2 68817 selfupdate (https://rclone.org/commands/rclone_selfupdate/) (Ivan 68818 Andreev) 68819 .RS 2 68820 .IP \[bu] 2 68821 Allows rclone to update itself in-place or via a package (using 68822 \f[C]--package\f[R] flag) 68823 .IP \[bu] 2 68824 Reads cryptographically signed signatures for non beta releases 68825 .IP \[bu] 2 68826 Works on all OSes. 68827 .RE 68828 .IP \[bu] 2 68829 test (https://rclone.org/commands/rclone_test/) - these are test 68830 commands - use with care! 68831 .RS 2 68832 .IP \[bu] 2 68833 \f[C]histogram\f[R] - Makes a histogram of file name characters. 68834 .IP \[bu] 2 68835 \f[C]info\f[R] - Discovers file name or other limitations for paths. 68836 .IP \[bu] 2 68837 \f[C]makefiles\f[R] - Make a random file hierarchy for testing. 68838 .IP \[bu] 2 68839 \f[C]memory\f[R] - Load all the objects at remote:path into memory and 68840 report memory stats. 68841 .RE 68842 .RE 68843 .IP \[bu] 2 68844 New Features 68845 .RS 2 68846 .IP \[bu] 2 68847 Connection strings (https://rclone.org/docs/#connection-strings) 68848 .RS 2 68849 .IP \[bu] 2 68850 Config parameters can now be passed as part of the remote name as a 68851 connection string. 68852 .IP \[bu] 2 68853 For example, to do the equivalent of \f[C]--drive-shared-with-me\f[R] 68854 use \f[C]drive,shared_with_me:\f[R] 68855 .IP \[bu] 2 68856 Make sure we don\[aq]t save on the fly remote config to the config file 68857 (Nick Craig-Wood) 68858 .IP \[bu] 2 68859 Make sure backends with additional config have a different name for 68860 caching (Nick Craig-Wood) 68861 .IP \[bu] 2 68862 This work was sponsored by CERN, through the CS3MESH4EOSC 68863 Project (https://cs3mesh4eosc.eu/). 68864 .RS 2 68865 .IP \[bu] 2 68866 CS3MESH4EOSC has received funding from the European Union\[cq]s Horizon 68867 2020 68868 .IP \[bu] 2 68869 research and innovation programme under Grant Agreement no. 68870 863353. 68871 .RE 68872 .RE 68873 .IP \[bu] 2 68874 build 68875 .RS 2 68876 .IP \[bu] 2 68877 Update go build version to go1.16 and raise minimum go version to go1.13 68878 (Nick Craig-Wood) 68879 .IP \[bu] 2 68880 Make a macOS ARM64 build to support Apple Silicon (Nick Craig-Wood) 68881 .IP \[bu] 2 68882 Install macfuse 4.x instead of osxfuse 3.x (Nick Craig-Wood) 68883 .IP \[bu] 2 68884 Use \f[C]GO386=softfloat\f[R] instead of deprecated \f[C]GO386=387\f[R] 68885 for 386 builds (Nick Craig-Wood) 68886 .IP \[bu] 2 68887 Disable IOS builds for the time being (Nick Craig-Wood) 68888 .IP \[bu] 2 68889 Androids builds made with up to date NDK (x0b) 68890 .IP \[bu] 2 68891 Add an rclone user to the Docker image but don\[aq]t use it by default 68892 (cynthia kwok) 68893 .RE 68894 .IP \[bu] 2 68895 dedupe: Make largest directory primary to minimize data moved (Saksham 68896 Khanna) 68897 .IP \[bu] 2 68898 config 68899 .RS 2 68900 .IP \[bu] 2 68901 Wrap config library in an interface (Fionera) 68902 .IP \[bu] 2 68903 Make config file system pluggable (Nick Craig-Wood) 68904 .IP \[bu] 2 68905 \f[C]--config \[dq]\[dq]\f[R] or \f[C]\[dq]/notfound\[dq]\f[R] for in 68906 memory config only (Nick Craig-Wood) 68907 .IP \[bu] 2 68908 Clear fs cache of stale entries when altering config (Nick Craig-Wood) 68909 .RE 68910 .IP \[bu] 2 68911 copyurl: Add option to print resulting auto-filename (albertony) 68912 .IP \[bu] 2 68913 delete: Make \f[C]--rmdirs\f[R] obey the filters (Nick Craig-Wood) 68914 .IP \[bu] 2 68915 docs - many fixes and reworks from edwardxml, albertony, pvalls, Ivan 68916 Andreev, Evan Harris, buengese, Alexey Tabakman 68917 .IP \[bu] 2 68918 encoder/filename - add SCSU as tables (Klaus Post) 68919 .IP \[bu] 2 68920 Add multiple paths support to \f[C]--compare-dest\f[R] and 68921 \f[C]--copy-dest\f[R] flag (K265) 68922 .IP \[bu] 2 68923 filter: Make \f[C]--exclude \[dq]dir/\[dq]\f[R] equivalent to 68924 \f[C]--exclude \[dq]dir/**\[dq]\f[R] (Nick Craig-Wood) 68925 .IP \[bu] 2 68926 fshttp: Add DSCP support with \f[C]--dscp\f[R] for QoS with 68927 differentiated services (Max Sum) 68928 .IP \[bu] 2 68929 lib/cache: Add Delete and DeletePrefix methods (Nick Craig-Wood) 68930 .IP \[bu] 2 68931 lib/file 68932 .RS 2 68933 .IP \[bu] 2 68934 Make pre-allocate detect disk full errors and return them (Nick 68935 Craig-Wood) 68936 .IP \[bu] 2 68937 Don\[aq]t run preallocate concurrently (Nick Craig-Wood) 68938 .IP \[bu] 2 68939 Retry preallocate on EINTR (Nick Craig-Wood) 68940 .RE 68941 .IP \[bu] 2 68942 operations: Made copy and sync operations obey a RetryAfterError (Ankur 68943 Gupta) 68944 .IP \[bu] 2 68945 rc 68946 .RS 2 68947 .IP \[bu] 2 68948 Add string alternatives for setting options over the rc (Nick 68949 Craig-Wood) 68950 .IP \[bu] 2 68951 Add \f[C]options/local\f[R] to see the options configured in the context 68952 (Nick Craig-Wood) 68953 .IP \[bu] 2 68954 Add \f[C]_config\f[R] parameter to set global config for just this rc 68955 call (Nick Craig-Wood) 68956 .IP \[bu] 2 68957 Implement passing filter config with \f[C]_filter\f[R] parameter (Nick 68958 Craig-Wood) 68959 .IP \[bu] 2 68960 Add \f[C]fscache/clear\f[R] and \f[C]fscache/entries\f[R] to control the 68961 fs cache (Nick Craig-Wood) 68962 .IP \[bu] 2 68963 Avoid +Inf value for speed in \f[C]core/stats\f[R] (albertony) 68964 .IP \[bu] 2 68965 Add a full set of stats to \f[C]core/stats\f[R] (Nick Craig-Wood) 68966 .IP \[bu] 2 68967 Allow \f[C]fs=\f[R] params to be a JSON blob (Nick Craig-Wood) 68968 .RE 68969 .IP \[bu] 2 68970 rcd: Added systemd notification during the \f[C]rclone rcd\f[R] command. 68971 (Naveen Honest Raj) 68972 .IP \[bu] 2 68973 rmdirs: Make \f[C]--rmdirs\f[R] obey the filters (Nick Craig-Wood) 68974 .IP \[bu] 2 68975 version: Show build tags and type of executable (Ivan Andreev) 68976 .RE 68977 .IP \[bu] 2 68978 Bug Fixes 68979 .RS 2 68980 .IP \[bu] 2 68981 install.sh: make it fail on download errors (Ivan Andreev) 68982 .IP \[bu] 2 68983 Fix excessive retries missing \f[C]--max-duration\f[R] timeout (Nick 68984 Craig-Wood) 68985 .IP \[bu] 2 68986 Fix crash when \f[C]--low-level-retries=0\f[R] (Nick Craig-Wood) 68987 .IP \[bu] 2 68988 Fix failed token refresh on mounts created via the rc (Nick Craig-Wood) 68989 .IP \[bu] 2 68990 fshttp: Fix bandwidth limiting after bad merge (Nick Craig-Wood) 68991 .IP \[bu] 2 68992 lib/atexit 68993 .RS 2 68994 .IP \[bu] 2 68995 Unregister interrupt handler once it has fired so users can interrupt 68996 again (Nick Craig-Wood) 68997 .IP \[bu] 2 68998 Fix occasional failure to unmount with CTRL-C (Nick Craig-Wood) 68999 .IP \[bu] 2 69000 Fix deadlock calling Finalise while Run is running (Nick Craig-Wood) 69001 .RE 69002 .IP \[bu] 2 69003 lib/rest: Fix multipart uploads not stopping on context cancel (Nick 69004 Craig-Wood) 69005 .RE 69006 .IP \[bu] 2 69007 Mount 69008 .RS 2 69009 .IP \[bu] 2 69010 Allow mounting to root directory on windows (albertony) 69011 .IP \[bu] 2 69012 Improved handling of relative paths on windows (albertony) 69013 .IP \[bu] 2 69014 Fix unicode issues with accented characters on macOS (Nick Craig-Wood) 69015 .IP \[bu] 2 69016 Docs: document the new FileSecurity option in WinFsp 2021 (albertony) 69017 .IP \[bu] 2 69018 Docs: add note about volume path syntax on windows (albertony) 69019 .IP \[bu] 2 69020 Fix caching of old directories after renaming them (Nick Craig-Wood) 69021 .IP \[bu] 2 69022 Update cgofuse to the latest version to bring in macfuse 4 fix (Nick 69023 Craig-Wood) 69024 .RE 69025 .IP \[bu] 2 69026 VFS 69027 .RS 2 69028 .IP \[bu] 2 69029 \f[C]--vfs-used-is-size\f[R] to report used space using recursive scan 69030 (tYYGH) 69031 .IP \[bu] 2 69032 Don\[aq]t set modification time if it was already correct (Nick 69033 Craig-Wood) 69034 .IP \[bu] 2 69035 Fix Create causing windows explorer to truncate files on CTRL-C CTRL-V 69036 (Nick Craig-Wood) 69037 .IP \[bu] 2 69038 Fix modtimes not updating when writing via cache (Nick Craig-Wood) 69039 .IP \[bu] 2 69040 Fix modtimes changing by fractional seconds after upload (Nick 69041 Craig-Wood) 69042 .IP \[bu] 2 69043 Fix modtime set if \f[C]--vfs-cache-mode writes\f[R]/\f[C]full\f[R] and 69044 no write (Nick Craig-Wood) 69045 .IP \[bu] 2 69046 Rename files in cache and cancel uploads on directory rename (Nick 69047 Craig-Wood) 69048 .IP \[bu] 2 69049 Fix directory renaming by renaming dirs cached in memory (Nick 69050 Craig-Wood) 69051 .RE 69052 .IP \[bu] 2 69053 Local 69054 .RS 2 69055 .IP \[bu] 2 69056 Add flag \f[C]--local-no-preallocate\f[R] (David Sze) 69057 .IP \[bu] 2 69058 Make \f[C]nounc\f[R] an advanced option except on Windows (albertony) 69059 .IP \[bu] 2 69060 Don\[aq]t ignore preallocate disk full errors (Nick Craig-Wood) 69061 .RE 69062 .IP \[bu] 2 69063 Cache 69064 .RS 2 69065 .IP \[bu] 2 69066 Add \f[C]--fs-cache-expire-duration\f[R] to control the fs cache (Nick 69067 Craig-Wood) 69068 .RE 69069 .IP \[bu] 2 69070 Crypt 69071 .RS 2 69072 .IP \[bu] 2 69073 Add option to not encrypt data (Vesnyx) 69074 .IP \[bu] 2 69075 Log hash ok on upload (albertony) 69076 .RE 69077 .IP \[bu] 2 69078 Azure Blob 69079 .RS 2 69080 .IP \[bu] 2 69081 Add container public access level support. 69082 (Manish Kumar) 69083 .RE 69084 .IP \[bu] 2 69085 B2 69086 .RS 2 69087 .IP \[bu] 2 69088 Fix HTML files downloaded via cloudflare (Nick Craig-Wood) 69089 .RE 69090 .IP \[bu] 2 69091 Box 69092 .RS 2 69093 .IP \[bu] 2 69094 Fix transfers getting stuck on token expiry after API change (Nick 69095 Craig-Wood) 69096 .RE 69097 .IP \[bu] 2 69098 Chunker 69099 .RS 2 69100 .IP \[bu] 2 69101 Partially implement no-rename transactions (Maxwell Calman) 69102 .RE 69103 .IP \[bu] 2 69104 Drive 69105 .RS 2 69106 .IP \[bu] 2 69107 Don\[aq]t stop server side copy if couldn\[aq]t read description (Nick 69108 Craig-Wood) 69109 .IP \[bu] 2 69110 Pass context on to drive SDK - to help with cancellation (Nick 69111 Craig-Wood) 69112 .RE 69113 .IP \[bu] 2 69114 Dropbox 69115 .RS 2 69116 .IP \[bu] 2 69117 Add polling for changes support (Robert Thomas) 69118 .IP \[bu] 2 69119 Make \f[C]--timeout 0\f[R] work properly (Nick Craig-Wood) 69120 .IP \[bu] 2 69121 Raise priority of rate limited message to INFO to make it more 69122 noticeable (Nick Craig-Wood) 69123 .RE 69124 .IP \[bu] 2 69125 Fichier 69126 .RS 2 69127 .IP \[bu] 2 69128 Implement copy & move (buengese) 69129 .IP \[bu] 2 69130 Implement public link (buengese) 69131 .RE 69132 .IP \[bu] 2 69133 FTP 69134 .RS 2 69135 .IP \[bu] 2 69136 Implement Shutdown method (Nick Craig-Wood) 69137 .IP \[bu] 2 69138 Close idle connections after \f[C]--ftp-idle-timeout\f[R] (1m by 69139 default) (Nick Craig-Wood) 69140 .IP \[bu] 2 69141 Make \f[C]--timeout 0\f[R] work properly (Nick Craig-Wood) 69142 .IP \[bu] 2 69143 Add \f[C]--ftp-close-timeout\f[R] flag for use with awkward ftp servers 69144 (Nick Craig-Wood) 69145 .IP \[bu] 2 69146 Retry connections and logins on 421 errors (Nick Craig-Wood) 69147 .RE 69148 .IP \[bu] 2 69149 Hdfs 69150 .RS 2 69151 .IP \[bu] 2 69152 Fix permissions for when directory is created (Lucas Messenger) 69153 .RE 69154 .IP \[bu] 2 69155 Onedrive 69156 .RS 2 69157 .IP \[bu] 2 69158 Make \f[C]--timeout 0\f[R] work properly (Nick Craig-Wood) 69159 .RE 69160 .IP \[bu] 2 69161 S3 69162 .RS 2 69163 .IP \[bu] 2 69164 Fix \f[C]--s3-profile\f[R] which wasn\[aq]t working (Nick Craig-Wood) 69165 .RE 69166 .IP \[bu] 2 69167 SFTP 69168 .RS 2 69169 .IP \[bu] 2 69170 Close idle connections after \f[C]--sftp-idle-timeout\f[R] (1m by 69171 default) (Nick Craig-Wood) 69172 .IP \[bu] 2 69173 Fix \[dq]file not found\[dq] errors for read once servers (Nick 69174 Craig-Wood) 69175 .IP \[bu] 2 69176 Fix SetModTime stat failed: object not found with 69177 \f[C]--sftp-set-modtime=false\f[R] (Nick Craig-Wood) 69178 .RE 69179 .IP \[bu] 2 69180 Swift 69181 .RS 2 69182 .IP \[bu] 2 69183 Update github.com/ncw/swift to v2.0.0 (Nick Craig-Wood) 69184 .IP \[bu] 2 69185 Implement copying large objects (nguyenhuuluan434) 69186 .RE 69187 .IP \[bu] 2 69188 Union 69189 .RS 2 69190 .IP \[bu] 2 69191 Fix crash when using epff policy (Nick Craig-Wood) 69192 .IP \[bu] 2 69193 Fix union attempting to update files on a read only file system (Nick 69194 Craig-Wood) 69195 .IP \[bu] 2 69196 Refactor to use fspath.SplitFs instead of fs.ParseRemote (Nick 69197 Craig-Wood) 69198 .IP \[bu] 2 69199 Fix initialisation broken in refactor (Nick Craig-Wood) 69200 .RE 69201 .IP \[bu] 2 69202 WebDAV 69203 .RS 2 69204 .IP \[bu] 2 69205 Add support for sharepoint with NTLM authentication (Rauno Ots) 69206 .IP \[bu] 2 69207 Make sharepoint-ntlm docs more consistent (Alex Chen) 69208 .IP \[bu] 2 69209 Improve terminology in sharepoint-ntlm docs (Ivan Andreev) 69210 .IP \[bu] 2 69211 Disable HTTP/2 for NTLM authentication (georne) 69212 .IP \[bu] 2 69213 Fix sharepoint-ntlm error 401 for parallel actions (Ivan Andreev) 69214 .IP \[bu] 2 69215 Check that purged directory really exists (Ivan Andreev) 69216 .RE 69217 .IP \[bu] 2 69218 Yandex 69219 .RS 2 69220 .IP \[bu] 2 69221 Make \f[C]--timeout 0\f[R] work properly (Nick Craig-Wood) 69222 .RE 69223 .IP \[bu] 2 69224 Zoho 69225 .RS 2 69226 .IP \[bu] 2 69227 Replace client id - you will need to \f[C]rclone config reconnect\f[R] 69228 after this (buengese) 69229 .IP \[bu] 2 69230 Add forgotten setupRegion() to NewFs - this finally fixes regions other 69231 than EU (buengese) 69232 .RE 69233 .SS v1.54.1 - 2021-03-08 69234 .PP 69235 See commits (https://github.com/artpar/artpar/compare/v1.54.0...v1.54.1) 69236 .IP \[bu] 2 69237 Bug Fixes 69238 .RS 2 69239 .IP \[bu] 2 69240 accounting: Fix --bwlimit when up or down is off (Nick Craig-Wood) 69241 .IP \[bu] 2 69242 docs 69243 .RS 2 69244 .IP \[bu] 2 69245 Fix nesting of brackets and backticks in ftp docs (edwardxml) 69246 .IP \[bu] 2 69247 Fix broken link in sftp page (edwardxml) 69248 .IP \[bu] 2 69249 Fix typo in crypt.md (Romeo Kienzler) 69250 .IP \[bu] 2 69251 Changelog: Correct link to digitalis.io (Alex JOST) 69252 .IP \[bu] 2 69253 Replace #file-caching with #vfs-file-caching (Miron Veryanskiy) 69254 .IP \[bu] 2 69255 Convert bogus example link to code (edwardxml) 69256 .IP \[bu] 2 69257 Remove dead link from rc.md (edwardxml) 69258 .RE 69259 .IP \[bu] 2 69260 rc: Sync,copy,move: document createEmptySrcDirs parameter (Nick 69261 Craig-Wood) 69262 .IP \[bu] 2 69263 lsjson: Fix unterminated JSON in the presence of errors (Nick 69264 Craig-Wood) 69265 .RE 69266 .IP \[bu] 2 69267 Mount 69268 .RS 2 69269 .IP \[bu] 2 69270 Fix mount dropping on macOS by setting --daemon-timeout 10m (Nick 69271 Craig-Wood) 69272 .RE 69273 .IP \[bu] 2 69274 VFS 69275 .RS 2 69276 .IP \[bu] 2 69277 Document simultaneous usage with the same cache shouldn\[aq]t be used 69278 (Nick Craig-Wood) 69279 .RE 69280 .IP \[bu] 2 69281 B2 69282 .RS 2 69283 .IP \[bu] 2 69284 Automatically raise upload cutoff to avoid spurious error (Nick 69285 Craig-Wood) 69286 .IP \[bu] 2 69287 Fix failed to create file system with application key limited to a 69288 prefix (Nick Craig-Wood) 69289 .RE 69290 .IP \[bu] 2 69291 Drive 69292 .RS 2 69293 .IP \[bu] 2 69294 Refer to Shared Drives instead of Team Drives (Nick Craig-Wood) 69295 .RE 69296 .IP \[bu] 2 69297 Dropbox 69298 .RS 2 69299 .IP \[bu] 2 69300 Add scopes to oauth request and optionally \[dq]members.read\[dq] (Nick 69301 Craig-Wood) 69302 .RE 69303 .IP \[bu] 2 69304 S3 69305 .RS 2 69306 .IP \[bu] 2 69307 Fix failed to create file system with folder level permissions policy 69308 (Nick Craig-Wood) 69309 .IP \[bu] 2 69310 Fix Wasabi HEAD requests returning stale data by using only 1 transport 69311 (Nick Craig-Wood) 69312 .IP \[bu] 2 69313 Fix shared_credentials_file auth (Dmitry Chepurovskiy) 69314 .IP \[bu] 2 69315 Add --s3-no-head to reducing costs docs (Nick Craig-Wood) 69316 .RE 69317 .IP \[bu] 2 69318 Union 69319 .RS 2 69320 .IP \[bu] 2 69321 Fix mkdir at root with remote:/ (Nick Craig-Wood) 69322 .RE 69323 .IP \[bu] 2 69324 Zoho 69325 .RS 2 69326 .IP \[bu] 2 69327 Fix custom client id\[aq]s (buengese) 69328 .RE 69329 .SS v1.54.0 - 2021-02-02 69330 .PP 69331 See commits (https://github.com/artpar/artpar/compare/v1.53.0...v1.54.0) 69332 .IP \[bu] 2 69333 New backends 69334 .RS 2 69335 .IP \[bu] 2 69336 Compression remote (experimental) (buengese) 69337 .IP \[bu] 2 69338 Enterprise File Fabric (Nick Craig-Wood) 69339 .RS 2 69340 .IP \[bu] 2 69341 This work was sponsored by Storage Made 69342 Easy (https://storagemadeeasy.com/) 69343 .RE 69344 .IP \[bu] 2 69345 HDFS (Hadoop Distributed File System) (Yury Stankevich) 69346 .IP \[bu] 2 69347 Zoho workdrive (buengese) 69348 .RE 69349 .IP \[bu] 2 69350 New Features 69351 .RS 2 69352 .IP \[bu] 2 69353 Deglobalise the config (Nick Craig-Wood) 69354 .RS 2 69355 .IP \[bu] 2 69356 Global config now read from the context 69357 .IP \[bu] 2 69358 This will enable passing of global config via the rc 69359 .IP \[bu] 2 69360 This work was sponsored by Digitalis (https://digitalis.io/) 69361 .RE 69362 .IP \[bu] 2 69363 Add \f[C]--bwlimit\f[R] for upload and download (Nick Craig-Wood) 69364 .RS 2 69365 .IP \[bu] 2 69366 Obey bwlimit in http Transport for better limiting 69367 .RE 69368 .IP \[bu] 2 69369 Enhance systemd integration (Hekmon) 69370 .RS 2 69371 .IP \[bu] 2 69372 log level identification, manual activation with flag, automatic systemd 69373 launch detection 69374 .IP \[bu] 2 69375 Don\[aq]t compile systemd log integration for non unix systems (Benjamin 69376 Gustin) 69377 .RE 69378 .IP \[bu] 2 69379 Add a \f[C]--download\f[R] flag to md5sum/sha1sum/hashsum to force 69380 rclone to download and hash files locally (lostheli) 69381 .IP \[bu] 2 69382 Add \f[C]--progress-terminal-title\f[R] to print ETA to terminal title 69383 (LaSombra) 69384 .IP \[bu] 2 69385 Make backend env vars show in help as the defaults for backend flags 69386 (Nick Craig-Wood) 69387 .IP \[bu] 2 69388 build 69389 .RS 2 69390 .IP \[bu] 2 69391 Raise minimum go version to go1.12 (Nick Craig-Wood) 69392 .RE 69393 .IP \[bu] 2 69394 dedupe 69395 .RS 2 69396 .IP \[bu] 2 69397 Add \f[C]--by-hash\f[R] to dedupe on content hash not file name (Nick 69398 Craig-Wood) 69399 .IP \[bu] 2 69400 Add \f[C]--dedupe-mode list\f[R] to just list dupes, changing nothing 69401 (Nick Craig-Wood) 69402 .IP \[bu] 2 69403 Add warning if used on a remote which can\[aq]t have duplicate names 69404 (Nick Craig-Wood) 69405 .RE 69406 .IP \[bu] 2 69407 fs 69408 .RS 2 69409 .IP \[bu] 2 69410 Add Shutdown optional method for backends (Nick Craig-Wood) 69411 .IP \[bu] 2 69412 When using \f[C]--files-from\f[R] check files concurrently (zhucan) 69413 .IP \[bu] 2 69414 Accumulate stats when using \f[C]--dry-run\f[R] (Ingo Weiss) 69415 .IP \[bu] 2 69416 Always show stats when using \f[C]--dry-run\f[R] or 69417 \f[C]--interactive\f[R] (Nick Craig-Wood) 69418 .IP \[bu] 2 69419 Add support for flag \f[C]--no-console\f[R] on windows to hide the 69420 console window (albertony) 69421 .RE 69422 .IP \[bu] 2 69423 genautocomplete: Add support to output to stdout (Ingo) 69424 .IP \[bu] 2 69425 ncdu 69426 .RS 2 69427 .IP \[bu] 2 69428 Highlight read errors instead of aborting (Claudio Bantaloukas) 69429 .IP \[bu] 2 69430 Add sort by average size in directory (Adam Pl\['a]nsk\['y]) 69431 .IP \[bu] 2 69432 Add toggle option for average s3ize in directory - key \[aq]a\[aq] (Adam 69433 Pl\['a]nsk\['y]) 69434 .IP \[bu] 2 69435 Add empty folder flag into ncdu browser (Adam Pl\['a]nsk\['y]) 69436 .IP \[bu] 2 69437 Add \f[C]!\f[R] (error) and \f[C].\f[R] (unreadable) file flags to go 69438 with \f[C]e\f[R] (empty) (Nick Craig-Wood) 69439 .RE 69440 .IP \[bu] 2 69441 obscure: Make \f[C]rclone obscure -\f[R] ignore newline at end of line 69442 (Nick Craig-Wood) 69443 .IP \[bu] 2 69444 operations 69445 .RS 2 69446 .IP \[bu] 2 69447 Add logs when need to upload files to set mod times (Nick Craig-Wood) 69448 .IP \[bu] 2 69449 Move and copy log name of the destination object in verbose (Adam 69450 Pl\['a]nsk\['y]) 69451 .IP \[bu] 2 69452 Add size if known to skipped items and JSON log (Nick Craig-Wood) 69453 .RE 69454 .IP \[bu] 2 69455 rc 69456 .RS 2 69457 .IP \[bu] 2 69458 Prefer actual listener address if using \[dq]:port\[dq] or 69459 \[dq]addr:0\[dq] only (Nick Craig-Wood) 69460 .IP \[bu] 2 69461 Add listener for finished jobs (Aleksandar Jankovic) 69462 .RE 69463 .IP \[bu] 2 69464 serve ftp: Add options to enable TLS (Deepak Sah) 69465 .IP \[bu] 2 69466 serve http/webdav: Redirect requests to the base url without the / (Nick 69467 Craig-Wood) 69468 .IP \[bu] 2 69469 serve restic: Implement object cache (Nick Craig-Wood) 69470 .IP \[bu] 2 69471 stats: Add counter for deleted directories (Nick Craig-Wood) 69472 .IP \[bu] 2 69473 sync: Only print \[dq]There was nothing to transfer\[dq] if no errors 69474 (Nick Craig-Wood) 69475 .IP \[bu] 2 69476 webui 69477 .RS 2 69478 .IP \[bu] 2 69479 Prompt user for updating webui if an update is available (Chaitanya 69480 Bankanhal) 69481 .IP \[bu] 2 69482 Fix plugins initialization (negative0) 69483 .RE 69484 .RE 69485 .IP \[bu] 2 69486 Bug Fixes 69487 .RS 2 69488 .IP \[bu] 2 69489 fs 69490 .RS 2 69491 .IP \[bu] 2 69492 Fix nil pointer on copy & move operations directly to remote (Anagh 69493 Kumar Baranwal) 69494 .IP \[bu] 2 69495 Fix parsing of .. 69496 when joining remotes (Nick Craig-Wood) 69497 .RE 69498 .IP \[bu] 2 69499 log: Fix enabling systemd logging when using \f[C]--log-file\f[R] (Nick 69500 Craig-Wood) 69501 .IP \[bu] 2 69502 check 69503 .RS 2 69504 .IP \[bu] 2 69505 Make the error count match up in the log message (Nick Craig-Wood) 69506 .RE 69507 .IP \[bu] 2 69508 move: Fix data loss when source and destination are the same object 69509 (Nick Craig-Wood) 69510 .IP \[bu] 2 69511 operations 69512 .RS 2 69513 .IP \[bu] 2 69514 Fix \f[C]--cutoff-mode\f[R] hard not cutting off immediately (Nick 69515 Craig-Wood) 69516 .IP \[bu] 2 69517 Fix \f[C]--immutable\f[R] error message (Nick Craig-Wood) 69518 .RE 69519 .IP \[bu] 2 69520 sync 69521 .RS 2 69522 .IP \[bu] 2 69523 Fix \f[C]--cutoff-mode\f[R] soft & cautious so it doesn\[aq]t end the 69524 transfer early (Nick Craig-Wood) 69525 .IP \[bu] 2 69526 Fix \f[C]--immutable\f[R] errors retrying many times (Nick Craig-Wood) 69527 .RE 69528 .RE 69529 .IP \[bu] 2 69530 Docs 69531 .RS 2 69532 .IP \[bu] 2 69533 Many fixes and a rewrite of the filtering docs (edwardxml) 69534 .IP \[bu] 2 69535 Many spelling and grammar fixes (Josh Soref) 69536 .IP \[bu] 2 69537 Doc fixes for commands delete, purge, rmdir, rmdirs and mount 69538 (albertony) 69539 .IP \[bu] 2 69540 And thanks to these people for many doc fixes too numerous to list 69541 .RS 2 69542 .IP \[bu] 2 69543 Ameer Dawood, Antoine GIRARD, Bob Bagwill, Christopher Stewart 69544 .IP \[bu] 2 69545 CokeMine, David, Dov Murik, Durval Menezes, Evan Harris, gtorelly 69546 .IP \[bu] 2 69547 Ilyess Bachiri, Janne Johansson, Kerry Su, Marcin Zelent, 69548 .IP \[bu] 2 69549 Martin Michlmayr, Milly, S\[u01A1]n Tr\[u1EA7]n-Nguy\[u1EC5]n 69550 .RE 69551 .RE 69552 .IP \[bu] 2 69553 Mount 69554 .RS 2 69555 .IP \[bu] 2 69556 Update systemd status with cache stats (Hekmon) 69557 .IP \[bu] 2 69558 Disable bazil/fuse based mount on macOS (Nick Craig-Wood) 69559 .RS 2 69560 .IP \[bu] 2 69561 Make \f[C]rclone mount\f[R] actually run \f[C]rclone cmount\f[R] under 69562 macOS (Nick Craig-Wood) 69563 .RE 69564 .IP \[bu] 2 69565 Implement mknod to make NFS file creation work (Nick Craig-Wood) 69566 .IP \[bu] 2 69567 Make sure we don\[aq]t call umount more than once (Nick Craig-Wood) 69568 .IP \[bu] 2 69569 More user friendly mounting as network drive on windows (albertony) 69570 .IP \[bu] 2 69571 Detect if uid or gid are set in same option string: -o uid=123,gid=456 69572 (albertony) 69573 .IP \[bu] 2 69574 Don\[aq]t attempt to unmount if fs has been destroyed already (Nick 69575 Craig-Wood) 69576 .RE 69577 .IP \[bu] 2 69578 VFS 69579 .RS 2 69580 .IP \[bu] 2 69581 Fix virtual entries causing deleted files to still appear (Nick 69582 Craig-Wood) 69583 .IP \[bu] 2 69584 Fix \[dq]file already exists\[dq] error for stale cache files (Nick 69585 Craig-Wood) 69586 .IP \[bu] 2 69587 Fix file leaks with \f[C]--vfs-cache-mode\f[R] full and 69588 \f[C]--buffer-size 0\f[R] (Nick Craig-Wood) 69589 .IP \[bu] 2 69590 Fix invalid cache path on windows when using :backend: as remote 69591 (albertony) 69592 .RE 69593 .IP \[bu] 2 69594 Local 69595 .RS 2 69596 .IP \[bu] 2 69597 Continue listing files/folders when a circular symlink is detected 69598 (Manish Gupta) 69599 .IP \[bu] 2 69600 New flag \f[C]--local-zero-size-links\f[R] to fix sync on some virtual 69601 filesystems (Riccardo Iaconelli) 69602 .RE 69603 .IP \[bu] 2 69604 Azure Blob 69605 .RS 2 69606 .IP \[bu] 2 69607 Add support for service principals (James Lim) 69608 .IP \[bu] 2 69609 Add support for managed identities (Brad Ackerman) 69610 .IP \[bu] 2 69611 Add examples for access tier (Bob Pusateri) 69612 .IP \[bu] 2 69613 Utilize the streaming capabilities from the SDK for multipart uploads 69614 (Denis Neuling) 69615 .IP \[bu] 2 69616 Fix setting of mime types (Nick Craig-Wood) 69617 .IP \[bu] 2 69618 Fix crash when listing outside a SAS URL\[aq]s root (Nick Craig-Wood) 69619 .IP \[bu] 2 69620 Delete archive tier blobs before update if 69621 \f[C]--azureblob-archive-tier-delete\f[R] (Nick Craig-Wood) 69622 .IP \[bu] 2 69623 Fix crash on startup (Nick Craig-Wood) 69624 .IP \[bu] 2 69625 Fix memory usage by upgrading the SDK to v0.13.0 and implementing a 69626 TransferManager (Nick Craig-Wood) 69627 .IP \[bu] 2 69628 Require go1.14+ to compile due to SDK changes (Nick Craig-Wood) 69629 .RE 69630 .IP \[bu] 2 69631 B2 69632 .RS 2 69633 .IP \[bu] 2 69634 Make NewObject use less expensive API calls (Nick Craig-Wood) 69635 .RS 2 69636 .IP \[bu] 2 69637 This will improve \f[C]--files-from\f[R] and \f[C]restic serve\f[R] in 69638 particular 69639 .RE 69640 .IP \[bu] 2 69641 Fixed crash on an empty file name (lluuaapp) 69642 .RE 69643 .IP \[bu] 2 69644 Box 69645 .RS 2 69646 .IP \[bu] 2 69647 Fix NewObject for files that differ in case (Nick Craig-Wood) 69648 .IP \[bu] 2 69649 Fix finding directories in a case insensitive way (Nick Craig-Wood) 69650 .RE 69651 .IP \[bu] 2 69652 Chunker 69653 .RS 2 69654 .IP \[bu] 2 69655 Skip long local hashing, hash in-transit (fixes) (Ivan Andreev) 69656 .IP \[bu] 2 69657 Set Features ReadMimeType to false as Object.MimeType not supported 69658 (Nick Craig-Wood) 69659 .IP \[bu] 2 69660 Fix case-insensitive NewObject, test metadata detection (Ivan Andreev) 69661 .RE 69662 .IP \[bu] 2 69663 Drive 69664 .RS 2 69665 .IP \[bu] 2 69666 Implement \f[C]rclone backend copyid\f[R] command for copying files by 69667 ID (Nick Craig-Wood) 69668 .IP \[bu] 2 69669 Added flag \f[C]--drive-stop-on-download-limit\f[R] to stop transfers 69670 when the download limit is exceeded (Anagh Kumar Baranwal) 69671 .IP \[bu] 2 69672 Implement CleanUp workaround for team drives (buengese) 69673 .IP \[bu] 2 69674 Allow shortcut resolution and creation to be retried (Nick Craig-Wood) 69675 .IP \[bu] 2 69676 Log that emptying the trash can take some time (Nick Craig-Wood) 69677 .IP \[bu] 2 69678 Add xdg office icons to xdg desktop files (Pau Rodriguez-Estivill) 69679 .RE 69680 .IP \[bu] 2 69681 Dropbox 69682 .RS 2 69683 .IP \[bu] 2 69684 Add support for viewing shared files and folders (buengese) 69685 .IP \[bu] 2 69686 Enable short lived access tokens (Nick Craig-Wood) 69687 .IP \[bu] 2 69688 Implement IDer on Objects so \f[C]rclone lsf\f[R] etc can read the IDs 69689 (buengese) 69690 .IP \[bu] 2 69691 Set Features ReadMimeType to false as Object.MimeType not supported 69692 (Nick Craig-Wood) 69693 .IP \[bu] 2 69694 Make malformed_path errors from too long files not retriable (Nick 69695 Craig-Wood) 69696 .IP \[bu] 2 69697 Test file name length before upload to fix upload loop (Nick Craig-Wood) 69698 .RE 69699 .IP \[bu] 2 69700 Fichier 69701 .RS 2 69702 .IP \[bu] 2 69703 Set Features ReadMimeType to true as Object.MimeType is supported (Nick 69704 Craig-Wood) 69705 .RE 69706 .IP \[bu] 2 69707 FTP 69708 .RS 2 69709 .IP \[bu] 2 69710 Add \f[C]--ftp-disable-msld\f[R] option to ignore MLSD for really old 69711 servers (Nick Craig-Wood) 69712 .IP \[bu] 2 69713 Make \f[C]--tpslimit apply\f[R] (Nick Craig-Wood) 69714 .RE 69715 .IP \[bu] 2 69716 Google Cloud Storage 69717 .RS 2 69718 .IP \[bu] 2 69719 Storage class object header support (Laurens Janssen) 69720 .IP \[bu] 2 69721 Fix anonymous client to use rclone\[aq]s HTTP client (Nick Craig-Wood) 69722 .IP \[bu] 2 69723 Fix 69724 \f[C]Entry doesn\[aq]t belong in directory \[dq]\[dq] (same as directory) - ignoring\f[R] 69725 (Nick Craig-Wood) 69726 .RE 69727 .IP \[bu] 2 69728 Googlephotos 69729 .RS 2 69730 .IP \[bu] 2 69731 New flag \f[C]--gphotos-include-archived\f[R] to show archived photos as 69732 well (Nicolas Rueff) 69733 .RE 69734 .IP \[bu] 2 69735 Jottacloud 69736 .RS 2 69737 .IP \[bu] 2 69738 Don\[aq]t erroneously report support for writing mime types (buengese) 69739 .IP \[bu] 2 69740 Add support for Telia Cloud (Patrik Nordl\['e]n) 69741 .RE 69742 .IP \[bu] 2 69743 Mailru 69744 .RS 2 69745 .IP \[bu] 2 69746 Accept special folders eg camera-upload (Ivan Andreev) 69747 .IP \[bu] 2 69748 Avoid prehashing of large local files (Ivan Andreev) 69749 .IP \[bu] 2 69750 Fix uploads after recent changes on server (Ivan Andreev) 69751 .IP \[bu] 2 69752 Fix range requests after June 2020 changes on server (Ivan Andreev) 69753 .IP \[bu] 2 69754 Fix invalid timestamp on corrupted files (fixes) (Ivan Andreev) 69755 .IP \[bu] 2 69756 Remove deprecated protocol quirks (Ivan Andreev) 69757 .RE 69758 .IP \[bu] 2 69759 Memory 69760 .RS 2 69761 .IP \[bu] 2 69762 Fix setting of mime types (Nick Craig-Wood) 69763 .RE 69764 .IP \[bu] 2 69765 Onedrive 69766 .RS 2 69767 .IP \[bu] 2 69768 Add support for China region operated by 21vianet and other regional 69769 suppliers (NyaMisty) 69770 .IP \[bu] 2 69771 Warn on gateway timeout errors (Nick Craig-Wood) 69772 .IP \[bu] 2 69773 Fall back to normal copy if server-side copy unavailable (Alex Chen) 69774 .IP \[bu] 2 69775 Fix server-side copy completely disabled on OneDrive for Business (Cnly) 69776 .IP \[bu] 2 69777 (business only) workaround to replace existing file on server-side copy 69778 (Alex Chen) 69779 .IP \[bu] 2 69780 Enhance link creation with expiry, scope, type and password (Nick 69781 Craig-Wood) 69782 .IP \[bu] 2 69783 Remove % and # from the set of encoded characters (Alex Chen) 69784 .IP \[bu] 2 69785 Support addressing site by server-relative URL (kice) 69786 .RE 69787 .IP \[bu] 2 69788 Opendrive 69789 .RS 2 69790 .IP \[bu] 2 69791 Fix finding directories in a case insensitive way (Nick Craig-Wood) 69792 .RE 69793 .IP \[bu] 2 69794 Pcloud 69795 .RS 2 69796 .IP \[bu] 2 69797 Fix setting of mime types (Nick Craig-Wood) 69798 .RE 69799 .IP \[bu] 2 69800 Premiumizeme 69801 .RS 2 69802 .IP \[bu] 2 69803 Fix finding directories in a case insensitive way (Nick Craig-Wood) 69804 .RE 69805 .IP \[bu] 2 69806 Qingstor 69807 .RS 2 69808 .IP \[bu] 2 69809 Fix error propagation in CleanUp (Nick Craig-Wood) 69810 .IP \[bu] 2 69811 Fix rclone cleanup (Nick Craig-Wood) 69812 .RE 69813 .IP \[bu] 2 69814 S3 69815 .RS 2 69816 .IP \[bu] 2 69817 Added \f[C]--s3-disable-http2\f[R] to disable http/2 (Anagh Kumar 69818 Baranwal) 69819 .IP \[bu] 2 69820 Complete SSE-C implementation (Nick Craig-Wood) 69821 .RS 2 69822 .IP \[bu] 2 69823 Fix hashes on small files with AWS:KMS and SSE-C (Nick Craig-Wood) 69824 .IP \[bu] 2 69825 Add MD5 metadata to objects uploaded with SSE-AWS/SSE-C (Nick 69826 Craig-Wood) 69827 .RE 69828 .IP \[bu] 2 69829 Add \f[C]--s3-no-head parameter\f[R] to minimise transactions on upload 69830 (Nick Craig-Wood) 69831 .IP \[bu] 2 69832 Update docs with a Reducing Costs section (Nick Craig-Wood) 69833 .IP \[bu] 2 69834 Added error handling for error code 429 indicating too many requests 69835 (Anagh Kumar Baranwal) 69836 .IP \[bu] 2 69837 Add requester pays option (kelv) 69838 .IP \[bu] 2 69839 Fix copy multipart with v2 auth failing with 69840 \[aq]SignatureDoesNotMatch\[aq] (Louis Koo) 69841 .RE 69842 .IP \[bu] 2 69843 SFTP 69844 .RS 2 69845 .IP \[bu] 2 69846 Allow cert based auth via optional pubkey (Stephen Harris) 69847 .IP \[bu] 2 69848 Allow user to optionally check server hosts key to add security (Stephen 69849 Harris) 69850 .IP \[bu] 2 69851 Defer asking for user passwords until the SSH connection succeeds 69852 (Stephen Harris) 69853 .IP \[bu] 2 69854 Remember entered password in AskPass mode (Stephen Harris) 69855 .IP \[bu] 2 69856 Implement Shutdown method (Nick Craig-Wood) 69857 .IP \[bu] 2 69858 Implement keyboard interactive authentication (Nick Craig-Wood) 69859 .IP \[bu] 2 69860 Make \f[C]--tpslimit\f[R] apply (Nick Craig-Wood) 69861 .IP \[bu] 2 69862 Implement \f[C]--sftp-use-fstat\f[R] for unusual SFTP servers (Nick 69863 Craig-Wood) 69864 .RE 69865 .IP \[bu] 2 69866 Sugarsync 69867 .RS 2 69868 .IP \[bu] 2 69869 Fix NewObject for files that differ in case (Nick Craig-Wood) 69870 .IP \[bu] 2 69871 Fix finding directories in a case insensitive way (Nick Craig-Wood) 69872 .RE 69873 .IP \[bu] 2 69874 Swift 69875 .RS 2 69876 .IP \[bu] 2 69877 Fix deletion of parts of Static Large Object (SLO) (Nguy\[u1EC5]n 69878 H\[u1EEF]u Lu\[^a]n) 69879 .IP \[bu] 2 69880 Ensure partially uploaded large files are uploaded unless 69881 \f[C]--swift-leave-parts-on-error\f[R] (Nguy\[u1EC5]n H\[u1EEF]u 69882 Lu\[^a]n) 69883 .RE 69884 .IP \[bu] 2 69885 Tardigrade 69886 .RS 2 69887 .IP \[bu] 2 69888 Upgrade to uplink v1.4.1 (Caleb Case) 69889 .RE 69890 .IP \[bu] 2 69891 WebDAV 69892 .RS 2 69893 .IP \[bu] 2 69894 Updated docs to show streaming to nextcloud is working (Durval Menezes) 69895 .RE 69896 .IP \[bu] 2 69897 Yandex 69898 .RS 2 69899 .IP \[bu] 2 69900 Set Features WriteMimeType to false as Yandex ignores mime types (Nick 69901 Craig-Wood) 69902 .RE 69903 .SS v1.53.4 - 2021-01-20 69904 .PP 69905 See commits (https://github.com/artpar/artpar/compare/v1.53.3...v1.53.4) 69906 .IP \[bu] 2 69907 Bug Fixes 69908 .RS 2 69909 .IP \[bu] 2 69910 accounting: Fix data race in Transferred() (Maciej Zimnoch) 69911 .IP \[bu] 2 69912 build 69913 .RS 2 69914 .IP \[bu] 2 69915 Stop tagged releases making a current beta (Nick Craig-Wood) 69916 .IP \[bu] 2 69917 Upgrade docker buildx action (Matteo Pietro Dazzi) 69918 .IP \[bu] 2 69919 Add -buildmode to cross-compile.go (Nick Craig-Wood) 69920 .IP \[bu] 2 69921 Fix docker build by upgrading ilteoood/docker_buildx (Nick Craig-Wood) 69922 .IP \[bu] 2 69923 Revert GitHub actions brew fix since this is now fixed (Nick Craig-Wood) 69924 .IP \[bu] 2 69925 Fix brew install --cask syntax for macOS build (Nick Craig-Wood) 69926 .IP \[bu] 2 69927 Update nfpm syntax to fix build of .deb/.rpm packages (Nick Craig-Wood) 69928 .IP \[bu] 2 69929 Fix for Windows build errors (Ivan Andreev) 69930 .RE 69931 .IP \[bu] 2 69932 fs: Parseduration: fixed tests to use UTC time (Ankur Gupta) 69933 .IP \[bu] 2 69934 fshttp: Prevent overlap of HTTP headers in logs (Nathan Collins) 69935 .IP \[bu] 2 69936 rc 69937 .RS 2 69938 .IP \[bu] 2 69939 Fix core/command giving 500 internal error (Nick Craig-Wood) 69940 .IP \[bu] 2 69941 Add Copy method to rc.Params (Nick Craig-Wood) 69942 .IP \[bu] 2 69943 Fix 500 error when marshalling errors from core/command (Nick 69944 Craig-Wood) 69945 .IP \[bu] 2 69946 plugins: Create plugins files only if webui is enabled. 69947 (negative0) 69948 .RE 69949 .IP \[bu] 2 69950 serve http: Fix serving files of unknown length (Nick Craig-Wood) 69951 .IP \[bu] 2 69952 serve sftp: Fix authentication on one connection blocking others (Nick 69953 Craig-Wood) 69954 .RE 69955 .IP \[bu] 2 69956 Mount 69957 .RS 2 69958 .IP \[bu] 2 69959 Add optional \f[C]brew\f[R] tag to throw an error when using mount in 69960 the binaries installed via Homebrew (Anagh Kumar Baranwal) 69961 .IP \[bu] 2 69962 Add \[dq].\[dq] and \[dq]..\[dq] to directories to match cmount and 69963 expectations (Nick Craig-Wood) 69964 .RE 69965 .IP \[bu] 2 69966 VFS 69967 .RS 2 69968 .IP \[bu] 2 69969 Make cache dir absolute before using it to fix path too long errors 69970 (Nick Craig-Wood) 69971 .RE 69972 .IP \[bu] 2 69973 Chunker 69974 .RS 2 69975 .IP \[bu] 2 69976 Improve detection of incompatible metadata (Ivan Andreev) 69977 .RE 69978 .IP \[bu] 2 69979 Google Cloud Storage 69980 .RS 2 69981 .IP \[bu] 2 69982 Fix server side copy of large objects (Nick Craig-Wood) 69983 .RE 69984 .IP \[bu] 2 69985 Jottacloud 69986 .RS 2 69987 .IP \[bu] 2 69988 Fix token renewer to fix long uploads (Nick Craig-Wood) 69989 .IP \[bu] 2 69990 Fix token refresh failed: is not a regular file error (Nick Craig-Wood) 69991 .RE 69992 .IP \[bu] 2 69993 Pcloud 69994 .RS 2 69995 .IP \[bu] 2 69996 Only use SHA1 hashes in EU region (Nick Craig-Wood) 69997 .RE 69998 .IP \[bu] 2 69999 Sharefile 70000 .RS 2 70001 .IP \[bu] 2 70002 Undo Fix backend due to API swapping integers for strings (Nick 70003 Craig-Wood) 70004 .RE 70005 .IP \[bu] 2 70006 WebDAV 70007 .RS 2 70008 .IP \[bu] 2 70009 Fix Open Range requests to fix 4shared mount (Nick Craig-Wood) 70010 .IP \[bu] 2 70011 Add \[dq]Depth: 0\[dq] to GET requests to fix bitrix (Nick Craig-Wood) 70012 .RE 70013 .SS v1.53.3 - 2020-11-19 70014 .PP 70015 See commits (https://github.com/artpar/artpar/compare/v1.53.2...v1.53.3) 70016 .IP \[bu] 2 70017 Bug Fixes 70018 .RS 2 70019 .IP \[bu] 2 70020 random: Fix incorrect use of math/rand instead of crypto/rand 70021 CVE-2020-28924 (Nick Craig-Wood) 70022 .RS 2 70023 .IP \[bu] 2 70024 Passwords you have generated with \f[C]rclone config\f[R] may be 70025 insecure 70026 .IP \[bu] 2 70027 See issue #4783 (https://github.com/artpar/artpar/issues/4783) for more 70028 details and a checking tool 70029 .RE 70030 .IP \[bu] 2 70031 random: Seed math/rand in one place with crypto strong seed (Nick 70032 Craig-Wood) 70033 .RE 70034 .IP \[bu] 2 70035 VFS 70036 .RS 2 70037 .IP \[bu] 2 70038 Fix vfs/refresh calls with fs= parameter (Nick Craig-Wood) 70039 .RE 70040 .IP \[bu] 2 70041 Sharefile 70042 .RS 2 70043 .IP \[bu] 2 70044 Fix backend due to API swapping integers for strings (Nick Craig-Wood) 70045 .RE 70046 .SS v1.53.2 - 2020-10-26 70047 .PP 70048 See commits (https://github.com/artpar/artpar/compare/v1.53.1...v1.53.2) 70049 .IP \[bu] 2 70050 Bug Fixes 70051 .RS 2 70052 .IP \[bu] 2 70053 accounting 70054 .RS 2 70055 .IP \[bu] 2 70056 Fix incorrect speed and transferTime in core/stats (Nick Craig-Wood) 70057 .IP \[bu] 2 70058 Stabilize display order of transfers on Windows (Nick Craig-Wood) 70059 .RE 70060 .IP \[bu] 2 70061 operations 70062 .RS 2 70063 .IP \[bu] 2 70064 Fix use of --suffix without --backup-dir (Nick Craig-Wood) 70065 .IP \[bu] 2 70066 Fix spurious \[dq]--checksum is in use but the source and destination 70067 have no hashes in common\[dq] (Nick Craig-Wood) 70068 .RE 70069 .IP \[bu] 2 70070 build 70071 .RS 2 70072 .IP \[bu] 2 70073 Work around GitHub actions brew problem (Nick Craig-Wood) 70074 .IP \[bu] 2 70075 Stop using set-env and set-path in the GitHub actions (Nick Craig-Wood) 70076 .RE 70077 .RE 70078 .IP \[bu] 2 70079 Mount 70080 .RS 2 70081 .IP \[bu] 2 70082 mount2: Fix the swapped UID / GID values (Russell Cattelan) 70083 .RE 70084 .IP \[bu] 2 70085 VFS 70086 .RS 2 70087 .IP \[bu] 2 70088 Detect and recover from a file being removed externally from the cache 70089 (Nick Craig-Wood) 70090 .IP \[bu] 2 70091 Fix a deadlock vulnerability in downloaders.Close (Leo Luan) 70092 .IP \[bu] 2 70093 Fix a race condition in retryFailedResets (Leo Luan) 70094 .IP \[bu] 2 70095 Fix missed concurrency control between some item operations and reset 70096 (Leo Luan) 70097 .IP \[bu] 2 70098 Add exponential backoff during ENOSPC retries (Leo Luan) 70099 .IP \[bu] 2 70100 Add a missed update of used cache space (Leo Luan) 70101 .IP \[bu] 2 70102 Fix --no-modtime to not attempt to set modtimes (as documented) (Nick 70103 Craig-Wood) 70104 .RE 70105 .IP \[bu] 2 70106 Local 70107 .RS 2 70108 .IP \[bu] 2 70109 Fix sizes and syncing with --links option on Windows (Nick Craig-Wood) 70110 .RE 70111 .IP \[bu] 2 70112 Chunker 70113 .RS 2 70114 .IP \[bu] 2 70115 Disable ListR to fix missing files on GDrive (workaround) (Ivan Andreev) 70116 .IP \[bu] 2 70117 Fix upload over crypt (Ivan Andreev) 70118 .RE 70119 .IP \[bu] 2 70120 Fichier 70121 .RS 2 70122 .IP \[bu] 2 70123 Increase maximum file size from 100GB to 300GB (gyutw) 70124 .RE 70125 .IP \[bu] 2 70126 Jottacloud 70127 .RS 2 70128 .IP \[bu] 2 70129 Remove clientSecret from config when upgrading to token based 70130 authentication (buengese) 70131 .IP \[bu] 2 70132 Avoid double url escaping of device/mountpoint (albertony) 70133 .IP \[bu] 2 70134 Remove DirMove workaround as it\[aq]s not required anymore - also 70135 (buengese) 70136 .RE 70137 .IP \[bu] 2 70138 Mailru 70139 .RS 2 70140 .IP \[bu] 2 70141 Fix uploads after recent changes on server (Ivan Andreev) 70142 .IP \[bu] 2 70143 Fix range requests after june changes on server (Ivan Andreev) 70144 .IP \[bu] 2 70145 Fix invalid timestamp on corrupted files (fixes) (Ivan Andreev) 70146 .RE 70147 .IP \[bu] 2 70148 Onedrive 70149 .RS 2 70150 .IP \[bu] 2 70151 Fix disk usage for sharepoint (Nick Craig-Wood) 70152 .RE 70153 .IP \[bu] 2 70154 S3 70155 .RS 2 70156 .IP \[bu] 2 70157 Add missing regions for AWS (Anagh Kumar Baranwal) 70158 .RE 70159 .IP \[bu] 2 70160 Seafile 70161 .RS 2 70162 .IP \[bu] 2 70163 Fix accessing libraries > 2GB on 32 bit systems (Muffin King) 70164 .RE 70165 .IP \[bu] 2 70166 SFTP 70167 .RS 2 70168 .IP \[bu] 2 70169 Always convert the checksum to lower case (buengese) 70170 .RE 70171 .IP \[bu] 2 70172 Union 70173 .RS 2 70174 .IP \[bu] 2 70175 Create root directories if none exist (Nick Craig-Wood) 70176 .RE 70177 .SS v1.53.1 - 2020-09-13 70178 .PP 70179 See commits (https://github.com/artpar/artpar/compare/v1.53.0...v1.53.1) 70180 .IP \[bu] 2 70181 Bug Fixes 70182 .RS 2 70183 .IP \[bu] 2 70184 accounting: Remove new line from end of --stats-one-line display (Nick 70185 Craig-Wood) 70186 .IP \[bu] 2 70187 check 70188 .RS 2 70189 .IP \[bu] 2 70190 Add back missing --download flag (Nick Craig-Wood) 70191 .IP \[bu] 2 70192 Fix docs (Nick Craig-Wood) 70193 .RE 70194 .IP \[bu] 2 70195 docs 70196 .RS 2 70197 .IP \[bu] 2 70198 Note --log-file does append (Nick Craig-Wood) 70199 .IP \[bu] 2 70200 Add full stops for consistency in rclone --help (edwardxml) 70201 .IP \[bu] 2 70202 Add Tencent COS to s3 provider list (wjielai) 70203 .IP \[bu] 2 70204 Updated mount command to reflect that it requires Go 1.13 or newer (Evan 70205 Harris) 70206 .IP \[bu] 2 70207 jottacloud: Mention that uploads from local disk will not need to cache 70208 files to disk for md5 calculation (albertony) 70209 .IP \[bu] 2 70210 Fix formatting of rc docs page (Nick Craig-Wood) 70211 .RE 70212 .IP \[bu] 2 70213 build 70214 .RS 2 70215 .IP \[bu] 2 70216 Include vendor tar ball in release and fix startdev (Nick Craig-Wood) 70217 .IP \[bu] 2 70218 Fix \[dq]Illegal instruction\[dq] error for ARMv6 builds (Nick 70219 Craig-Wood) 70220 .IP \[bu] 2 70221 Fix architecture name in ARMv7 build (Nick Craig-Wood) 70222 .RE 70223 .RE 70224 .IP \[bu] 2 70225 VFS 70226 .RS 2 70227 .IP \[bu] 2 70228 Fix spurious error \[dq]vfs cache: failed to _ensure cache EOF\[dq] 70229 (Nick Craig-Wood) 70230 .IP \[bu] 2 70231 Log an ERROR if we fail to set the file to be sparse (Nick Craig-Wood) 70232 .RE 70233 .IP \[bu] 2 70234 Local 70235 .RS 2 70236 .IP \[bu] 2 70237 Log an ERROR if we fail to set the file to be sparse (Nick Craig-Wood) 70238 .RE 70239 .IP \[bu] 2 70240 Drive 70241 .RS 2 70242 .IP \[bu] 2 70243 Re-adds special oauth help text (Tim Gallant) 70244 .RE 70245 .IP \[bu] 2 70246 Opendrive 70247 .RS 2 70248 .IP \[bu] 2 70249 Do not retry 400 errors (Evan Harris) 70250 .RE 70251 .SS v1.53.0 - 2020-09-02 70252 .PP 70253 See commits (https://github.com/artpar/artpar/compare/v1.52.0...v1.53.0) 70254 .IP \[bu] 2 70255 New Features 70256 .RS 2 70257 .IP \[bu] 2 70258 The VFS 70259 layer (https://rclone.org/commands/rclone_mount/#vfs-virtual-file-system) 70260 was heavily reworked for this release - see below for more details 70261 .IP \[bu] 2 70262 Interactive mode -i/--interactive (https://rclone.org/docs/#interactive) 70263 for destructive operations (fishbullet) 70264 .IP \[bu] 2 70265 Add 70266 --bwlimit-file (https://rclone.org/docs/#bwlimit-file-bandwidth-spec) 70267 flag to limit speeds of individual file transfers (Nick Craig-Wood) 70268 .IP \[bu] 2 70269 Transfers are sorted by start time in the stats and progress output (Max 70270 Sum) 70271 .IP \[bu] 2 70272 Make sure backends expand \f[C]\[ti]\f[R] and environment vars in file 70273 names they use (Nick Craig-Wood) 70274 .IP \[bu] 2 70275 Add --refresh-times (https://rclone.org/docs/#refresh-times) flag to set 70276 modtimes on hashless backends (Nick Craig-Wood) 70277 .IP \[bu] 2 70278 build 70279 .RS 2 70280 .IP \[bu] 2 70281 Remove vendor directory in favour of Go modules (Nick Craig-Wood) 70282 .IP \[bu] 2 70283 Build with go1.15.x by default (Nick Craig-Wood) 70284 .IP \[bu] 2 70285 Drop macOS 386 build as it is no longer supported by go1.15 (Nick 70286 Craig-Wood) 70287 .IP \[bu] 2 70288 Add ARMv7 to the supported builds (Nick Craig-Wood) 70289 .IP \[bu] 2 70290 Enable \f[C]rclone cmount\f[R] on macOS (Nick Craig-Wood) 70291 .IP \[bu] 2 70292 Make rclone build with gccgo (Nick Craig-Wood) 70293 .IP \[bu] 2 70294 Make rclone build with wasm (Nick Craig-Wood) 70295 .IP \[bu] 2 70296 Change beta numbering to be semver compatible (Nick Craig-Wood) 70297 .IP \[bu] 2 70298 Add file properties and icon to Windows executable (albertony) 70299 .IP \[bu] 2 70300 Add experimental interface for integrating rclone into browsers (Nick 70301 Craig-Wood) 70302 .RE 70303 .IP \[bu] 2 70304 lib: Add file name compression (Klaus Post) 70305 .IP \[bu] 2 70306 rc 70307 .RS 2 70308 .IP \[bu] 2 70309 Allow installation and use of plugins and test plugins with rclone-webui 70310 (Chaitanya Bankanhal) 70311 .IP \[bu] 2 70312 Add reverse proxy pluginsHandler for serving plugins (Chaitanya 70313 Bankanhal) 70314 .IP \[bu] 2 70315 Add \f[C]mount/listmounts\f[R] option for listing current mounts 70316 (Chaitanya Bankanhal) 70317 .IP \[bu] 2 70318 Add \f[C]operations/uploadfile\f[R] to upload a file through rc using 70319 encoding multipart/form-data (Chaitanya Bankanhal) 70320 .IP \[bu] 2 70321 Add \f[C]core/command\f[R] to execute rclone terminal commands. 70322 (Chaitanya Bankanhal) 70323 .RE 70324 .IP \[bu] 2 70325 \f[C]rclone check\f[R] 70326 .RS 2 70327 .IP \[bu] 2 70328 Add reporting of filenames for same/missing/changed (Nick Craig-Wood) 70329 .IP \[bu] 2 70330 Make check command obey 70331 \f[C]--dry-run\f[R]/\f[C]-i\f[R]/\f[C]--interactive\f[R] (Nick 70332 Craig-Wood) 70333 .IP \[bu] 2 70334 Make check do \f[C]--checkers\f[R] files concurrently (Nick Craig-Wood) 70335 .IP \[bu] 2 70336 Retry downloads if they fail when using the \f[C]--download\f[R] flag 70337 (Nick Craig-Wood) 70338 .IP \[bu] 2 70339 Make it show stats by default (Nick Craig-Wood) 70340 .RE 70341 .IP \[bu] 2 70342 \f[C]rclone obscure\f[R]: Allow obscure command to accept password on 70343 STDIN (David Ibarra) 70344 .IP \[bu] 2 70345 \f[C]rclone config\f[R] 70346 .RS 2 70347 .IP \[bu] 2 70348 Set RCLONE_CONFIG_DIR for use in config files and subprocesses (Nick 70349 Craig-Wood) 70350 .IP \[bu] 2 70351 Reject remote names starting with a dash. 70352 (jtagcat) 70353 .RE 70354 .IP \[bu] 2 70355 \f[C]rclone cryptcheck\f[R]: Add reporting of filenames for 70356 same/missing/changed (Nick Craig-Wood) 70357 .IP \[bu] 2 70358 \f[C]rclone dedupe\f[R]: Make it obey the \f[C]--size-only\f[R] flag for 70359 duplicate detection (Nick Craig-Wood) 70360 .IP \[bu] 2 70361 \f[C]rclone link\f[R]: Add \f[C]--expire\f[R] and \f[C]--unlink\f[R] 70362 flags (Roman Kredentser) 70363 .IP \[bu] 2 70364 \f[C]rclone mkdir\f[R]: Warn when using mkdir on remotes which can\[aq]t 70365 have empty directories (Nick Craig-Wood) 70366 .IP \[bu] 2 70367 \f[C]rclone rc\f[R]: Allow JSON parameters to simplify command line 70368 usage (Nick Craig-Wood) 70369 .IP \[bu] 2 70370 \f[C]rclone serve ftp\f[R] 70371 .RS 2 70372 .IP \[bu] 2 70373 Don\[aq]t compile on < go1.13 after dependency update (Nick Craig-Wood) 70374 .IP \[bu] 2 70375 Add error message if auth proxy fails (Nick Craig-Wood) 70376 .IP \[bu] 2 70377 Use refactored goftp.io/server library for binary shrink (Nick 70378 Craig-Wood) 70379 .RE 70380 .IP \[bu] 2 70381 \f[C]rclone serve restic\f[R]: Expose interfaces so that rclone can be 70382 used as a library from within restic (Jack) 70383 .IP \[bu] 2 70384 \f[C]rclone sync\f[R]: Add \f[C]--track-renames-strategy leaf\f[R] (Nick 70385 Craig-Wood) 70386 .IP \[bu] 2 70387 \f[C]rclone touch\f[R]: Add ability to set nanosecond resolution times 70388 (Nick Craig-Wood) 70389 .IP \[bu] 2 70390 \f[C]rclone tree\f[R]: Remove \f[C]-i\f[R] shorthand for 70391 \f[C]--noindent\f[R] as it conflicts with 70392 \f[C]-i\f[R]/\f[C]--interactive\f[R] (Nick Craig-Wood) 70393 .RE 70394 .IP \[bu] 2 70395 Bug Fixes 70396 .RS 2 70397 .IP \[bu] 2 70398 accounting 70399 .RS 2 70400 .IP \[bu] 2 70401 Fix documentation for \f[C]speed\f[R]/\f[C]speedAvg\f[R] (Nick 70402 Craig-Wood) 70403 .IP \[bu] 2 70404 Fix elapsed time not show actual time since beginning (Chaitanya 70405 Bankanhal) 70406 .IP \[bu] 2 70407 Fix deadlock in stats printing (Nick Craig-Wood) 70408 .RE 70409 .IP \[bu] 2 70410 build 70411 .RS 2 70412 .IP \[bu] 2 70413 Fix file handle leak in GitHub release tool (Garrett Squire) 70414 .RE 70415 .IP \[bu] 2 70416 \f[C]rclone check\f[R]: Fix successful retries with \f[C]--download\f[R] 70417 counting errors (Nick Craig-Wood) 70418 .IP \[bu] 2 70419 \f[C]rclone dedupe\f[R]: Fix logging to be easier to understand (Nick 70420 Craig-Wood) 70421 .RE 70422 .IP \[bu] 2 70423 Mount 70424 .RS 2 70425 .IP \[bu] 2 70426 Warn macOS users that mount implementation is changing (Nick Craig-Wood) 70427 .RS 2 70428 .IP \[bu] 2 70429 to test the new implementation use \f[C]rclone cmount\f[R] instead of 70430 \f[C]rclone mount\f[R] 70431 .IP \[bu] 2 70432 this is because the library rclone uses has dropped macOS support 70433 .RE 70434 .IP \[bu] 2 70435 rc interface 70436 .RS 2 70437 .IP \[bu] 2 70438 Add call for unmount all (Chaitanya Bankanhal) 70439 .IP \[bu] 2 70440 Make \f[C]mount/mount\f[R] remote control take vfsOpt option (Nick 70441 Craig-Wood) 70442 .IP \[bu] 2 70443 Add mountOpt to \f[C]mount/mount\f[R] (Nick Craig-Wood) 70444 .IP \[bu] 2 70445 Add VFS and Mount options to \f[C]mount/listmounts\f[R] (Nick 70446 Craig-Wood) 70447 .RE 70448 .IP \[bu] 2 70449 Catch panics in cgofuse initialization and turn into error messages 70450 (Nick Craig-Wood) 70451 .IP \[bu] 2 70452 Always supply stat information in Readdir (Nick Craig-Wood) 70453 .IP \[bu] 2 70454 Add support for reading unknown length files using direct IO (Windows) 70455 (Nick Craig-Wood) 70456 .IP \[bu] 2 70457 Fix On Windows don\[aq]t add \f[C]-o uid/gid=-1\f[R] if user supplies 70458 \f[C]-o uid/gid\f[R]. 70459 (Nick Craig-Wood) 70460 .IP \[bu] 2 70461 Fix macOS losing directory contents in cmount (Nick Craig-Wood) 70462 .IP \[bu] 2 70463 Fix volume name broken in recent refactor (Nick Craig-Wood) 70464 .RE 70465 .IP \[bu] 2 70466 VFS 70467 .RS 2 70468 .IP \[bu] 2 70469 Implement partial reads for \f[C]--vfs-cache-mode full\f[R] (Nick 70470 Craig-Wood) 70471 .IP \[bu] 2 70472 Add \f[C]--vfs-writeback\f[R] option to delay writes back to cloud 70473 storage (Nick Craig-Wood) 70474 .IP \[bu] 2 70475 Add \f[C]--vfs-read-ahead\f[R] parameter for use with 70476 \f[C]--vfs-cache-mode full\f[R] (Nick Craig-Wood) 70477 .IP \[bu] 2 70478 Restart pending uploads on restart of the cache (Nick Craig-Wood) 70479 .IP \[bu] 2 70480 Support synchronous cache space recovery upon ENOSPC (Leo Luan) 70481 .IP \[bu] 2 70482 Allow ReadAt and WriteAt to run concurrently with themselves (Nick 70483 Craig-Wood) 70484 .IP \[bu] 2 70485 Change modtime of file before upload to current (Rob Calistri) 70486 .IP \[bu] 2 70487 Recommend \f[C]--vfs-cache-modes writes\f[R] on backends which can\[aq]t 70488 stream (Nick Craig-Wood) 70489 .IP \[bu] 2 70490 Add an optional \f[C]fs\f[R] parameter to vfs rc methods (Nick 70491 Craig-Wood) 70492 .IP \[bu] 2 70493 Fix errors when using > 260 char files in the cache in Windows (Nick 70494 Craig-Wood) 70495 .IP \[bu] 2 70496 Fix renaming of items while they are being uploaded (Nick Craig-Wood) 70497 .IP \[bu] 2 70498 Fix very high load caused by slow directory listings (Nick Craig-Wood) 70499 .IP \[bu] 2 70500 Fix renamed files not being uploaded with 70501 \f[C]--vfs-cache-mode minimal\f[R] (Nick Craig-Wood) 70502 .IP \[bu] 2 70503 Fix directory locking caused by slow directory listings (Nick 70504 Craig-Wood) 70505 .IP \[bu] 2 70506 Fix saving from chrome without \f[C]--vfs-cache-mode writes\f[R] (Nick 70507 Craig-Wood) 70508 .RE 70509 .IP \[bu] 2 70510 Local 70511 .RS 2 70512 .IP \[bu] 2 70513 Add \f[C]--local-no-updated\f[R] to provide a consistent view of 70514 changing objects (Nick Craig-Wood) 70515 .IP \[bu] 2 70516 Add \f[C]--local-no-set-modtime\f[R] option to prevent modtime changes 70517 (tyhuber1) 70518 .IP \[bu] 2 70519 Fix race conditions updating and reading Object metadata (Nick 70520 Craig-Wood) 70521 .RE 70522 .IP \[bu] 2 70523 Cache 70524 .RS 2 70525 .IP \[bu] 2 70526 Make any created backends be cached to fix rc problems (Nick Craig-Wood) 70527 .IP \[bu] 2 70528 Fix dedupe on caches wrapping drives (Nick Craig-Wood) 70529 .RE 70530 .IP \[bu] 2 70531 Crypt 70532 .RS 2 70533 .IP \[bu] 2 70534 Add \f[C]--crypt-server-side-across-configs\f[R] flag (Nick Craig-Wood) 70535 .IP \[bu] 2 70536 Make any created backends be cached to fix rc problems (Nick Craig-Wood) 70537 .RE 70538 .IP \[bu] 2 70539 Alias 70540 .RS 2 70541 .IP \[bu] 2 70542 Make any created backends be cached to fix rc problems (Nick Craig-Wood) 70543 .RE 70544 .IP \[bu] 2 70545 Azure Blob 70546 .RS 2 70547 .IP \[bu] 2 70548 Don\[aq]t compile on < go1.13 after dependency update (Nick Craig-Wood) 70549 .RE 70550 .IP \[bu] 2 70551 B2 70552 .RS 2 70553 .IP \[bu] 2 70554 Implement server-side copy for files > 5GB (Nick Craig-Wood) 70555 .IP \[bu] 2 70556 Cancel in progress multipart uploads and copies on rclone exit (Nick 70557 Craig-Wood) 70558 .IP \[bu] 2 70559 Note that b2\[aq]s encoding now allows \ but rclone\[aq]s hasn\[aq]t 70560 changed (Nick Craig-Wood) 70561 .IP \[bu] 2 70562 Fix transfers when using download_url (Nick Craig-Wood) 70563 .RE 70564 .IP \[bu] 2 70565 Box 70566 .RS 2 70567 .IP \[bu] 2 70568 Implement rclone cleanup (buengese) 70569 .IP \[bu] 2 70570 Cancel in progress multipart uploads and copies on rclone exit (Nick 70571 Craig-Wood) 70572 .IP \[bu] 2 70573 Allow authentication with access token (David) 70574 .RE 70575 .IP \[bu] 2 70576 Chunker 70577 .RS 2 70578 .IP \[bu] 2 70579 Make any created backends be cached to fix rc problems (Nick Craig-Wood) 70580 .RE 70581 .IP \[bu] 2 70582 Drive 70583 .RS 2 70584 .IP \[bu] 2 70585 Add \f[C]rclone backend drives\f[R] to list shared drives (teamdrives) 70586 (Nick Craig-Wood) 70587 .IP \[bu] 2 70588 Implement \f[C]rclone backend untrash\f[R] (Nick Craig-Wood) 70589 .IP \[bu] 2 70590 Work around drive bug which didn\[aq]t set modtime of copied docs (Nick 70591 Craig-Wood) 70592 .IP \[bu] 2 70593 Added \f[C]--drive-starred-only\f[R] to only show starred files (Jay 70594 McEntire) 70595 .IP \[bu] 2 70596 Deprecate \f[C]--drive-alternate-export\f[R] as it is no longer needed 70597 (themylogin) 70598 .IP \[bu] 2 70599 Fix duplication of Google docs on server-side copy (Nick Craig-Wood) 70600 .IP \[bu] 2 70601 Fix \[dq]panic: send on closed channel\[dq] when recycling dir entries 70602 (Nick Craig-Wood) 70603 .RE 70604 .IP \[bu] 2 70605 Dropbox 70606 .RS 2 70607 .IP \[bu] 2 70608 Add copyright detector info in limitations section in the docs (Alex 70609 Guerrero) 70610 .IP \[bu] 2 70611 Fix \f[C]rclone link\f[R] by removing expires parameter (Nick 70612 Craig-Wood) 70613 .RE 70614 .IP \[bu] 2 70615 Fichier 70616 .RS 2 70617 .IP \[bu] 2 70618 Detect Flood detected: IP Locked error and sleep for 30s (Nick 70619 Craig-Wood) 70620 .RE 70621 .IP \[bu] 2 70622 FTP 70623 .RS 2 70624 .IP \[bu] 2 70625 Add explicit TLS support (Heiko Bornholdt) 70626 .IP \[bu] 2 70627 Add support for \f[C]--dump bodies\f[R] and \f[C]--dump auth\f[R] for 70628 debugging (Nick Craig-Wood) 70629 .IP \[bu] 2 70630 Fix interoperation with pure-ftpd (Nick Craig-Wood) 70631 .RE 70632 .IP \[bu] 2 70633 Google Cloud Storage 70634 .RS 2 70635 .IP \[bu] 2 70636 Add support for anonymous access (Kai L\[:u]ke) 70637 .RE 70638 .IP \[bu] 2 70639 Jottacloud 70640 .RS 2 70641 .IP \[bu] 2 70642 Bring back legacy authentication for use with whitelabel versions 70643 (buengese) 70644 .IP \[bu] 2 70645 Switch to new api root - also implement a very ugly workaround for the 70646 DirMove failures (buengese) 70647 .RE 70648 .IP \[bu] 2 70649 Onedrive 70650 .RS 2 70651 .IP \[bu] 2 70652 Rework cancel of multipart uploads on rclone exit (Nick Craig-Wood) 70653 .IP \[bu] 2 70654 Implement rclone cleanup (Nick Craig-Wood) 70655 .IP \[bu] 2 70656 Add \f[C]--onedrive-no-versions\f[R] flag to remove old versions (Nick 70657 Craig-Wood) 70658 .RE 70659 .IP \[bu] 2 70660 Pcloud 70661 .RS 2 70662 .IP \[bu] 2 70663 Implement \f[C]rclone link\f[R] for public link creation (buengese) 70664 .RE 70665 .IP \[bu] 2 70666 Qingstor 70667 .RS 2 70668 .IP \[bu] 2 70669 Cancel in progress multipart uploads on rclone exit (Nick Craig-Wood) 70670 .RE 70671 .IP \[bu] 2 70672 S3 70673 .RS 2 70674 .IP \[bu] 2 70675 Preserve metadata when doing multipart copy (Nick Craig-Wood) 70676 .IP \[bu] 2 70677 Cancel in progress multipart uploads and copies on rclone exit (Nick 70678 Craig-Wood) 70679 .IP \[bu] 2 70680 Add \f[C]rclone link\f[R] for public link sharing (Roman Kredentser) 70681 .IP \[bu] 2 70682 Add \f[C]rclone backend restore\f[R] command to restore objects from 70683 GLACIER (Nick Craig-Wood) 70684 .IP \[bu] 2 70685 Add \f[C]rclone cleanup\f[R] and \f[C]rclone backend cleanup\f[R] to 70686 clean unfinished multipart uploads (Nick Craig-Wood) 70687 .IP \[bu] 2 70688 Add \f[C]rclone backend list-multipart-uploads\f[R] to list unfinished 70689 multipart uploads (Nick Craig-Wood) 70690 .IP \[bu] 2 70691 Add \f[C]--s3-max-upload-parts\f[R] support (Kamil Trzci\[u0144]ski) 70692 .IP \[bu] 2 70693 Add \f[C]--s3-no-check-bucket\f[R] for minimising rclone transactions 70694 and perms (Nick Craig-Wood) 70695 .IP \[bu] 2 70696 Add \f[C]--s3-profile\f[R] and \f[C]--s3-shared-credentials-file\f[R] 70697 options (Nick Craig-Wood) 70698 .IP \[bu] 2 70699 Use regional s3 us-east-1 endpoint (David) 70700 .IP \[bu] 2 70701 Add Scaleway provider (Vincent Feltz) 70702 .IP \[bu] 2 70703 Update IBM COS endpoints (Egor Margineanu) 70704 .IP \[bu] 2 70705 Reduce the default \f[C]--s3-copy-cutoff\f[R] to < 5GB for Backblaze S3 70706 compatibility (Nick Craig-Wood) 70707 .IP \[bu] 2 70708 Fix detection of bucket existing (Nick Craig-Wood) 70709 .RE 70710 .IP \[bu] 2 70711 SFTP 70712 .RS 2 70713 .IP \[bu] 2 70714 Use the absolute path instead of the relative path for listing for 70715 improved compatibility (Nick Craig-Wood) 70716 .IP \[bu] 2 70717 Add \f[C]--sftp-subsystem\f[R] and \f[C]--sftp-server-command\f[R] 70718 options (aus) 70719 .RE 70720 .IP \[bu] 2 70721 Swift 70722 .RS 2 70723 .IP \[bu] 2 70724 Fix dangling large objects breaking the listing (Nick Craig-Wood) 70725 .IP \[bu] 2 70726 Fix purge not deleting directory markers (Nick Craig-Wood) 70727 .IP \[bu] 2 70728 Fix update multipart object removing all of its own parts (Nick 70729 Craig-Wood) 70730 .IP \[bu] 2 70731 Fix missing hash from object returned from upload (Nick Craig-Wood) 70732 .RE 70733 .IP \[bu] 2 70734 Tardigrade 70735 .RS 2 70736 .IP \[bu] 2 70737 Upgrade to uplink v1.2.0 (Kaloyan Raev) 70738 .RE 70739 .IP \[bu] 2 70740 Union 70741 .RS 2 70742 .IP \[bu] 2 70743 Fix writing with the all policy (Nick Craig-Wood) 70744 .RE 70745 .IP \[bu] 2 70746 WebDAV 70747 .RS 2 70748 .IP \[bu] 2 70749 Fix directory creation with 4shared (Nick Craig-Wood) 70750 .RE 70751 .SS v1.52.3 - 2020-08-07 70752 .PP 70753 See commits (https://github.com/artpar/artpar/compare/v1.52.2...v1.52.3) 70754 .IP \[bu] 2 70755 Bug Fixes 70756 .RS 2 70757 .IP \[bu] 2 70758 docs 70759 .RS 2 70760 .IP \[bu] 2 70761 Disable smart typography (e.g. 70762 en-dash) in MANUAL.* and man page (Nick Craig-Wood) 70763 .IP \[bu] 2 70764 Update install.md to reflect minimum Go version (Evan Harris) 70765 .IP \[bu] 2 70766 Update install from source instructions (Nick Craig-Wood) 70767 .IP \[bu] 2 70768 make_manual: Support SOURCE_DATE_EPOCH (Morten Linderud) 70769 .RE 70770 .IP \[bu] 2 70771 log: Fix --use-json-log going to stderr not --log-file on Windows (Nick 70772 Craig-Wood) 70773 .IP \[bu] 2 70774 serve dlna: Fix file list on Samsung Series 6+ TVs (Matteo Pietro Dazzi) 70775 .IP \[bu] 2 70776 sync: Fix deadlock with --track-renames-strategy modtime (Nick 70777 Craig-Wood) 70778 .RE 70779 .IP \[bu] 2 70780 Cache 70781 .RS 2 70782 .IP \[bu] 2 70783 Fix moveto/copyto remote:file remote:file2 (Nick Craig-Wood) 70784 .RE 70785 .IP \[bu] 2 70786 Drive 70787 .RS 2 70788 .IP \[bu] 2 70789 Stop using root_folder_id as a cache (Nick Craig-Wood) 70790 .IP \[bu] 2 70791 Make dangling shortcuts appear in listings (Nick Craig-Wood) 70792 .IP \[bu] 2 70793 Drop \[dq]Disabling ListR\[dq] messages down to debug (Nick Craig-Wood) 70794 .IP \[bu] 2 70795 Workaround and policy for Google Drive API (Dmitry Ustalov) 70796 .RE 70797 .IP \[bu] 2 70798 FTP 70799 .RS 2 70800 .IP \[bu] 2 70801 Add note to docs about home vs root directory selection (Nick 70802 Craig-Wood) 70803 .RE 70804 .IP \[bu] 2 70805 Onedrive 70806 .RS 2 70807 .IP \[bu] 2 70808 Fix reverting to Copy when Move would have worked (Nick Craig-Wood) 70809 .IP \[bu] 2 70810 Avoid comma rendered in URL in onedrive.md (Kevin) 70811 .RE 70812 .IP \[bu] 2 70813 Pcloud 70814 .RS 2 70815 .IP \[bu] 2 70816 Fix oauth on European region \[dq]eapi.pcloud.com\[dq] (Nick Craig-Wood) 70817 .RE 70818 .IP \[bu] 2 70819 S3 70820 .RS 2 70821 .IP \[bu] 2 70822 Fix bucket Region auto detection when Region unset in config (Nick 70823 Craig-Wood) 70824 .RE 70825 .SS v1.52.2 - 2020-06-24 70826 .PP 70827 See commits (https://github.com/artpar/artpar/compare/v1.52.1...v1.52.2) 70828 .IP \[bu] 2 70829 Bug Fixes 70830 .RS 2 70831 .IP \[bu] 2 70832 build 70833 .RS 2 70834 .IP \[bu] 2 70835 Fix docker release build action (Nick Craig-Wood) 70836 .IP \[bu] 2 70837 Fix custom timezone in Docker image (NoLooseEnds) 70838 .RE 70839 .IP \[bu] 2 70840 check: Fix misleading message which printed errors instead of 70841 differences (Nick Craig-Wood) 70842 .IP \[bu] 2 70843 errors: Add WSAECONNREFUSED and more to the list of retriable Windows 70844 errors (Nick Craig-Wood) 70845 .IP \[bu] 2 70846 rcd: Fix incorrect prometheus metrics (Gary Kim) 70847 .IP \[bu] 2 70848 serve restic: Fix flags so they use environment variables (Nick 70849 Craig-Wood) 70850 .IP \[bu] 2 70851 serve webdav: Fix flags so they use environment variables (Nick 70852 Craig-Wood) 70853 .IP \[bu] 2 70854 sync: Fix --track-renames-strategy modtime (Nick Craig-Wood) 70855 .RE 70856 .IP \[bu] 2 70857 Drive 70858 .RS 2 70859 .IP \[bu] 2 70860 Fix not being able to delete a directory with a trashed shortcut (Nick 70861 Craig-Wood) 70862 .IP \[bu] 2 70863 Fix creating a directory inside a shortcut (Nick Craig-Wood) 70864 .IP \[bu] 2 70865 Fix --drive-impersonate with cached root_folder_id (Nick Craig-Wood) 70866 .RE 70867 .IP \[bu] 2 70868 SFTP 70869 .RS 2 70870 .IP \[bu] 2 70871 Fix SSH key PEM loading (Zac Rubin) 70872 .RE 70873 .IP \[bu] 2 70874 Swift 70875 .RS 2 70876 .IP \[bu] 2 70877 Speed up deletes by not retrying segment container deletes (Nick 70878 Craig-Wood) 70879 .RE 70880 .IP \[bu] 2 70881 Tardigrade 70882 .RS 2 70883 .IP \[bu] 2 70884 Upgrade to uplink v1.1.1 (Caleb Case) 70885 .RE 70886 .IP \[bu] 2 70887 WebDAV 70888 .RS 2 70889 .IP \[bu] 2 70890 Fix free/used display for rclone about/df for certain backends (Nick 70891 Craig-Wood) 70892 .RE 70893 .SS v1.52.1 - 2020-06-10 70894 .PP 70895 See commits (https://github.com/artpar/artpar/compare/v1.52.0...v1.52.1) 70896 .IP \[bu] 2 70897 Bug Fixes 70898 .RS 2 70899 .IP \[bu] 2 70900 lib/file: Fix SetSparse on Windows 7 which fixes downloads of files > 70901 250MB (Nick Craig-Wood) 70902 .IP \[bu] 2 70903 build 70904 .RS 2 70905 .IP \[bu] 2 70906 Update go.mod to go1.14 to enable -mod=vendor build (Nick Craig-Wood) 70907 .IP \[bu] 2 70908 Remove quicktest from Dockerfile (Nick Craig-Wood) 70909 .IP \[bu] 2 70910 Build Docker images with GitHub actions (Matteo Pietro Dazzi) 70911 .IP \[bu] 2 70912 Update Docker build workflows (Nick Craig-Wood) 70913 .IP \[bu] 2 70914 Set user_allow_other in /etc/fuse.conf in the Docker image (Nick 70915 Craig-Wood) 70916 .IP \[bu] 2 70917 Fix xgo build after go1.14 go.mod update (Nick Craig-Wood) 70918 .RE 70919 .IP \[bu] 2 70920 docs 70921 .RS 2 70922 .IP \[bu] 2 70923 Add link to source and modified time to footer of every page (Nick 70924 Craig-Wood) 70925 .IP \[bu] 2 70926 Remove manually set dates and use git dates instead (Nick Craig-Wood) 70927 .IP \[bu] 2 70928 Minor tense, punctuation, brevity and positivity changes for the home 70929 page (edwardxml) 70930 .IP \[bu] 2 70931 Remove leading slash in page reference in footer when present (Nick 70932 Craig-Wood) 70933 .IP \[bu] 2 70934 Note commands which need obscured input in the docs (Nick Craig-Wood) 70935 .IP \[bu] 2 70936 obscure: Write more help as we are referencing it elsewhere (Nick 70937 Craig-Wood) 70938 .RE 70939 .RE 70940 .IP \[bu] 2 70941 VFS 70942 .RS 2 70943 .IP \[bu] 2 70944 Fix OS vs Unix path confusion - fixes ChangeNotify on Windows (Nick 70945 Craig-Wood) 70946 .RE 70947 .IP \[bu] 2 70948 Drive 70949 .RS 2 70950 .IP \[bu] 2 70951 Fix missing items when listing using --fast-list / ListR (Nick 70952 Craig-Wood) 70953 .RE 70954 .IP \[bu] 2 70955 Putio 70956 .RS 2 70957 .IP \[bu] 2 70958 Fix panic on Object.Open (Cenk Alti) 70959 .RE 70960 .IP \[bu] 2 70961 S3 70962 .RS 2 70963 .IP \[bu] 2 70964 Fix upload of single files into buckets without create permission (Nick 70965 Craig-Wood) 70966 .IP \[bu] 2 70967 Fix --header-upload (Nick Craig-Wood) 70968 .RE 70969 .IP \[bu] 2 70970 Tardigrade 70971 .RS 2 70972 .IP \[bu] 2 70973 Fix listing bug by upgrading to v1.0.7 70974 .IP \[bu] 2 70975 Set UserAgent to rclone (Caleb Case) 70976 .RE 70977 .SS v1.52.0 - 2020-05-27 70978 .PP 70979 Special thanks to Martin Michlmayr for proof reading and correcting all 70980 the docs and Edward Barker for helping re-write the front page. 70981 .PP 70982 See commits (https://github.com/artpar/artpar/compare/v1.51.0...v1.52.0) 70983 .IP \[bu] 2 70984 New backends 70985 .RS 2 70986 .IP \[bu] 2 70987 Tardigrade (https://rclone.org/tardigrade/) backend for use with 70988 storj.io (Caleb Case) 70989 .IP \[bu] 2 70990 Union (https://rclone.org/union/) re-write to have multiple writable 70991 remotes (Max Sum) 70992 .IP \[bu] 2 70993 Seafile for Seafile server (Fred \[at]creativeprojects) 70994 .RE 70995 .IP \[bu] 2 70996 New commands 70997 .RS 2 70998 .IP \[bu] 2 70999 backend: command for backend-specific commands (see backends) (Nick 71000 Craig-Wood) 71001 .IP \[bu] 2 71002 cachestats: Deprecate in favour of \f[C]rclone backend stats cache:\f[R] 71003 (Nick Craig-Wood) 71004 .IP \[bu] 2 71005 dbhashsum: Deprecate in favour of \f[C]rclone hashsum DropboxHash\f[R] 71006 (Nick Craig-Wood) 71007 .RE 71008 .IP \[bu] 2 71009 New Features 71010 .RS 2 71011 .IP \[bu] 2 71012 Add \f[C]--header-download\f[R] and \f[C]--header-upload\f[R] flags for 71013 setting HTTP headers when uploading/downloading (Tim Gallant) 71014 .IP \[bu] 2 71015 Add \f[C]--header\f[R] flag to add HTTP headers to every HTTP 71016 transaction (Nick Craig-Wood) 71017 .IP \[bu] 2 71018 Add \f[C]--check-first\f[R] to do all checking before starting transfers 71019 (Nick Craig-Wood) 71020 .IP \[bu] 2 71021 Add \f[C]--track-renames-strategy\f[R] for configurable matching 71022 criteria for \f[C]--track-renames\f[R] (Bernd Schoolmann) 71023 .IP \[bu] 2 71024 Add \f[C]--cutoff-mode\f[R] hard,soft,cautious (Shing Kit Chan & 71025 Franklyn Tackitt) 71026 .IP \[bu] 2 71027 Filter flags (e.g. 71028 \f[C]--files-from -\f[R]) can read from stdin (fishbullet) 71029 .IP \[bu] 2 71030 Add \f[C]--error-on-no-transfer\f[R] option (Jon Fautley) 71031 .IP \[bu] 2 71032 Implement \f[C]--order-by xxx,mixed\f[R] for copying some small and some 71033 big files (Nick Craig-Wood) 71034 .IP \[bu] 2 71035 Allow \f[C]--max-backlog\f[R] to be negative meaning as large as 71036 possible (Nick Craig-Wood) 71037 .IP \[bu] 2 71038 Added \f[C]--no-unicode-normalization\f[R] flag to allow Unicode 71039 filenames to remain unique (Ben Zenker) 71040 .IP \[bu] 2 71041 Allow \f[C]--min-age\f[R]/\f[C]--max-age\f[R] to take a date as well as 71042 a duration (Nick Craig-Wood) 71043 .IP \[bu] 2 71044 Add rename statistics for file and directory renames (Nick Craig-Wood) 71045 .IP \[bu] 2 71046 Add statistics output to JSON log (reddi) 71047 .IP \[bu] 2 71048 Make stats be printed on non-zero exit code (Nick Craig-Wood) 71049 .IP \[bu] 2 71050 When running \f[C]--password-command\f[R] allow use of stdin 71051 (S\['e]bastien Gross) 71052 .IP \[bu] 2 71053 Stop empty strings being a valid remote path (Nick Craig-Wood) 71054 .IP \[bu] 2 71055 accounting: support WriterTo for less memory copying (Nick Craig-Wood) 71056 .IP \[bu] 2 71057 build 71058 .RS 2 71059 .IP \[bu] 2 71060 Update to use go1.14 for the build (Nick Craig-Wood) 71061 .IP \[bu] 2 71062 Add \f[C]-trimpath\f[R] to release build for reproduceable builds (Nick 71063 Craig-Wood) 71064 .IP \[bu] 2 71065 Remove GOOS and GOARCH from Dockerfile (Brandon Philips) 71066 .RE 71067 .IP \[bu] 2 71068 config 71069 .RS 2 71070 .IP \[bu] 2 71071 Fsync the config file after writing to save more reliably (Nick 71072 Craig-Wood) 71073 .IP \[bu] 2 71074 Add \f[C]--obscure\f[R] and \f[C]--no-obscure\f[R] flags to 71075 \f[C]config create\f[R]/\f[C]update\f[R] (Nick Craig-Wood) 71076 .IP \[bu] 2 71077 Make \f[C]config show\f[R] take \f[C]remote:\f[R] as well as 71078 \f[C]remote\f[R] (Nick Craig-Wood) 71079 .RE 71080 .IP \[bu] 2 71081 copyurl: Add \f[C]--no-clobber\f[R] flag (Denis) 71082 .IP \[bu] 2 71083 delete: Added \f[C]--rmdirs\f[R] flag to delete directories as well 71084 (Kush) 71085 .IP \[bu] 2 71086 filter: Added \f[C]--files-from-raw\f[R] flag (Ankur Gupta) 71087 .IP \[bu] 2 71088 genautocomplete: Add support for fish shell (Matan Rosenberg) 71089 .IP \[bu] 2 71090 log: Add support for syslog LOCAL facilities (Patryk Jakuszew) 71091 .IP \[bu] 2 71092 lsjson: Add \f[C]--hash-type\f[R] parameter and use it in lsf to speed 71093 up hashing (Nick Craig-Wood) 71094 .IP \[bu] 2 71095 rc 71096 .RS 2 71097 .IP \[bu] 2 71098 Add \f[C]-o\f[R]/\f[C]--opt\f[R] and \f[C]-a\f[R]/\f[C]--arg\f[R] for 71099 more structured input (Nick Craig-Wood) 71100 .IP \[bu] 2 71101 Implement \f[C]backend/command\f[R] for running backend-specific 71102 commands remotely (Nick Craig-Wood) 71103 .IP \[bu] 2 71104 Add \f[C]mount/mount\f[R] command for starting \f[C]rclone mount\f[R] 71105 via the API (Chaitanya) 71106 .RE 71107 .IP \[bu] 2 71108 rcd: Add Prometheus metrics support (Gary Kim) 71109 .IP \[bu] 2 71110 serve http 71111 .RS 2 71112 .IP \[bu] 2 71113 Added a \f[C]--template\f[R] flag for user defined markup (calistri) 71114 .IP \[bu] 2 71115 Add Last-Modified headers to files and directories (Nick Craig-Wood) 71116 .RE 71117 .IP \[bu] 2 71118 serve sftp: Add support for multiple host keys by repeating 71119 \f[C]--key\f[R] flag (Maxime Suret) 71120 .IP \[bu] 2 71121 touch: Add \f[C]--localtime\f[R] flag to make \f[C]--timestamp\f[R] 71122 localtime not UTC (Nick Craig-Wood) 71123 .RE 71124 .IP \[bu] 2 71125 Bug Fixes 71126 .RS 2 71127 .IP \[bu] 2 71128 accounting 71129 .RS 2 71130 .IP \[bu] 2 71131 Restore \[dq]Max number of stats groups reached\[dq] log line 71132 (Micha\[/l] Matczuk) 71133 .IP \[bu] 2 71134 Correct exitcode on Transfer Limit Exceeded flag. 71135 (Anuar Serdaliyev) 71136 .IP \[bu] 2 71137 Reset bytes read during copy retry (Ankur Gupta) 71138 .IP \[bu] 2 71139 Fix race clearing stats (Nick Craig-Wood) 71140 .RE 71141 .IP \[bu] 2 71142 copy: Only create empty directories when they don\[aq]t exist on the 71143 remote (Ishuah Kariuki) 71144 .IP \[bu] 2 71145 dedupe: Stop dedupe deleting files with identical IDs (Nick Craig-Wood) 71146 .IP \[bu] 2 71147 oauth 71148 .RS 2 71149 .IP \[bu] 2 71150 Use custom http client so that \f[C]--no-check-certificate\f[R] is 71151 honored by oauth token fetch (Mark Spieth) 71152 .IP \[bu] 2 71153 Replace deprecated oauth2.NoContext (Lars Lehtonen) 71154 .RE 71155 .IP \[bu] 2 71156 operations 71157 .RS 2 71158 .IP \[bu] 2 71159 Fix setting the timestamp on Windows for multithread copy (Nick 71160 Craig-Wood) 71161 .IP \[bu] 2 71162 Make rcat obey \f[C]--ignore-checksum\f[R] (Nick Craig-Wood) 71163 .IP \[bu] 2 71164 Make \f[C]--max-transfer\f[R] more accurate (Nick Craig-Wood) 71165 .RE 71166 .IP \[bu] 2 71167 rc 71168 .RS 2 71169 .IP \[bu] 2 71170 Fix dropped error (Lars Lehtonen) 71171 .IP \[bu] 2 71172 Fix misplaced http server config (Xiaoxing Ye) 71173 .IP \[bu] 2 71174 Disable duplicate log (ElonH) 71175 .RE 71176 .IP \[bu] 2 71177 serve dlna 71178 .RS 2 71179 .IP \[bu] 2 71180 Cds: don\[aq]t specify childCount at all when unknown (Dan Walters) 71181 .IP \[bu] 2 71182 Cds: use modification time as date in dlna metadata (Dan Walters) 71183 .RE 71184 .IP \[bu] 2 71185 serve restic: Fix tests after restic project removed vendoring (Nick 71186 Craig-Wood) 71187 .IP \[bu] 2 71188 sync 71189 .RS 2 71190 .IP \[bu] 2 71191 Fix incorrect \[dq]nothing to transfer\[dq] message using 71192 \f[C]--delete-before\f[R] (Nick Craig-Wood) 71193 .IP \[bu] 2 71194 Only create empty directories when they don\[aq]t exist on the remote 71195 (Ishuah Kariuki) 71196 .RE 71197 .RE 71198 .IP \[bu] 2 71199 Mount 71200 .RS 2 71201 .IP \[bu] 2 71202 Add \f[C]--async-read\f[R] flag to disable asynchronous reads (Nick 71203 Craig-Wood) 71204 .IP \[bu] 2 71205 Ignore \f[C]--allow-root\f[R] flag with a warning as it has been removed 71206 upstream (Nick Craig-Wood) 71207 .IP \[bu] 2 71208 Warn if \f[C]--allow-non-empty\f[R] used on Windows and clarify docs 71209 (Nick Craig-Wood) 71210 .IP \[bu] 2 71211 Constrain to go1.13 or above otherwise bazil.org/fuse fails to compile 71212 (Nick Craig-Wood) 71213 .IP \[bu] 2 71214 Fix fail because of too long volume name (evileye) 71215 .IP \[bu] 2 71216 Report 1PB free for unknown disk sizes (Nick Craig-Wood) 71217 .IP \[bu] 2 71218 Map more rclone errors into file systems errors (Nick Craig-Wood) 71219 .IP \[bu] 2 71220 Fix disappearing cwd problem (Nick Craig-Wood) 71221 .IP \[bu] 2 71222 Use ReaddirPlus on Windows to improve directory listing performance 71223 (Nick Craig-Wood) 71224 .IP \[bu] 2 71225 Send a hint as to whether the filesystem is case insensitive or not 71226 (Nick Craig-Wood) 71227 .IP \[bu] 2 71228 Add rc command \f[C]mount/types\f[R] (Nick Craig-Wood) 71229 .IP \[bu] 2 71230 Change maximum leaf name length to 1024 bytes (Nick Craig-Wood) 71231 .RE 71232 .IP \[bu] 2 71233 VFS 71234 .RS 2 71235 .IP \[bu] 2 71236 Add \f[C]--vfs-read-wait\f[R] and \f[C]--vfs-write-wait\f[R] flags to 71237 control time waiting for a sequential read/write (Nick Craig-Wood) 71238 .IP \[bu] 2 71239 Change default \f[C]--vfs-read-wait\f[R] to 20ms (it was 5ms and not 71240 configurable) (Nick Craig-Wood) 71241 .IP \[bu] 2 71242 Make \f[C]df\f[R] output more consistent on a rclone mount. 71243 (Yves G) 71244 .IP \[bu] 2 71245 Report 1PB free for unknown disk sizes (Nick Craig-Wood) 71246 .IP \[bu] 2 71247 Fix race condition caused by unlocked reading of Dir.path (Nick 71248 Craig-Wood) 71249 .IP \[bu] 2 71250 Make File lock and Dir lock not overlap to avoid deadlock (Nick 71251 Craig-Wood) 71252 .IP \[bu] 2 71253 Implement lock ordering between File and Dir to eliminate deadlocks 71254 (Nick Craig-Wood) 71255 .IP \[bu] 2 71256 Factor the vfs cache into its own package (Nick Craig-Wood) 71257 .IP \[bu] 2 71258 Pin the Fs in use in the Fs cache (Nick Craig-Wood) 71259 .IP \[bu] 2 71260 Add SetSys() methods to Node to allow caching stuff on a node (Nick 71261 Craig-Wood) 71262 .IP \[bu] 2 71263 Ignore file not found errors from Hash in Read.Release (Nick Craig-Wood) 71264 .IP \[bu] 2 71265 Fix hang in read wait code (Nick Craig-Wood) 71266 .RE 71267 .IP \[bu] 2 71268 Local 71269 .RS 2 71270 .IP \[bu] 2 71271 Speed up multi thread downloads by using sparse files on Windows (Nick 71272 Craig-Wood) 71273 .IP \[bu] 2 71274 Implement \f[C]--local-no-sparse\f[R] flag for disabling sparse files 71275 (Nick Craig-Wood) 71276 .IP \[bu] 2 71277 Implement \f[C]rclone backend noop\f[R] for testing purposes (Nick 71278 Craig-Wood) 71279 .IP \[bu] 2 71280 Fix \[dq]file not found\[dq] errors on post transfer Hash calculation 71281 (Nick Craig-Wood) 71282 .RE 71283 .IP \[bu] 2 71284 Cache 71285 .RS 2 71286 .IP \[bu] 2 71287 Implement \f[C]rclone backend stats\f[R] command (Nick Craig-Wood) 71288 .IP \[bu] 2 71289 Fix Server Side Copy with Temp Upload (Brandon McNama) 71290 .IP \[bu] 2 71291 Remove Unused Functions (Lars Lehtonen) 71292 .IP \[bu] 2 71293 Disable race tests until bbolt is fixed (Nick Craig-Wood) 71294 .IP \[bu] 2 71295 Move methods used for testing into test file (greatroar) 71296 .IP \[bu] 2 71297 Add Pin and Unpin and canonicalised lookup (Nick Craig-Wood) 71298 .IP \[bu] 2 71299 Use proper import path go.etcd.io/bbolt (Robert-Andr\['e] Mauchin) 71300 .RE 71301 .IP \[bu] 2 71302 Crypt 71303 .RS 2 71304 .IP \[bu] 2 71305 Calculate hashes for uploads from local disk (Nick Craig-Wood) 71306 .RS 2 71307 .IP \[bu] 2 71308 This allows encrypted Jottacloud uploads without using local disk 71309 .IP \[bu] 2 71310 This means encrypted s3/b2 uploads will now have hashes 71311 .RE 71312 .IP \[bu] 2 71313 Added \f[C]rclone backend decode\f[R]/\f[C]encode\f[R] commands to 71314 replicate functionality of \f[C]cryptdecode\f[R] (Anagh Kumar Baranwal) 71315 .IP \[bu] 2 71316 Get rid of the unused Cipher interface as it obfuscated the code (Nick 71317 Craig-Wood) 71318 .RE 71319 .IP \[bu] 2 71320 Azure Blob 71321 .RS 2 71322 .IP \[bu] 2 71323 Implement streaming of unknown sized files so \f[C]rcat\f[R] is now 71324 supported (Nick Craig-Wood) 71325 .IP \[bu] 2 71326 Implement memory pooling to control memory use (Nick Craig-Wood) 71327 .IP \[bu] 2 71328 Add \f[C]--azureblob-disable-checksum\f[R] flag (Nick Craig-Wood) 71329 .IP \[bu] 2 71330 Retry \f[C]InvalidBlobOrBlock\f[R] error as it may indicate block 71331 concurrency problems (Nick Craig-Wood) 71332 .IP \[bu] 2 71333 Remove unused \f[C]Object.parseTimeString()\f[R] (Lars Lehtonen) 71334 .IP \[bu] 2 71335 Fix permission error on SAS URL limited to container (Nick Craig-Wood) 71336 .RE 71337 .IP \[bu] 2 71338 B2 71339 .RS 2 71340 .IP \[bu] 2 71341 Add support for \f[C]--header-upload\f[R] and 71342 \f[C]--header-download\f[R] (Tim Gallant) 71343 .IP \[bu] 2 71344 Ignore directory markers at the root also (Nick Craig-Wood) 71345 .IP \[bu] 2 71346 Force the case of the SHA1 to lowercase (Nick Craig-Wood) 71347 .IP \[bu] 2 71348 Remove unused \f[C]largeUpload.clearUploadURL()\f[R] (Lars Lehtonen) 71349 .RE 71350 .IP \[bu] 2 71351 Box 71352 .RS 2 71353 .IP \[bu] 2 71354 Add support for \f[C]--header-upload\f[R] and 71355 \f[C]--header-download\f[R] (Tim Gallant) 71356 .IP \[bu] 2 71357 Implement About to read size used (Nick Craig-Wood) 71358 .IP \[bu] 2 71359 Add token renew function for jwt auth (David Bramwell) 71360 .IP \[bu] 2 71361 Added support for interchangeable root folder for Box backend (Sunil 71362 Patra) 71363 .IP \[bu] 2 71364 Remove unnecessary iat from jws claims (David) 71365 .RE 71366 .IP \[bu] 2 71367 Drive 71368 .RS 2 71369 .IP \[bu] 2 71370 Follow shortcuts by default, skip with \f[C]--drive-skip-shortcuts\f[R] 71371 (Nick Craig-Wood) 71372 .IP \[bu] 2 71373 Implement \f[C]rclone backend shortcut\f[R] command for creating 71374 shortcuts (Nick Craig-Wood) 71375 .IP \[bu] 2 71376 Added \f[C]rclone backend\f[R] command to change 71377 \f[C]service_account_file\f[R] and \f[C]chunk_size\f[R] (Anagh Kumar 71378 Baranwal) 71379 .IP \[bu] 2 71380 Fix missing files when using \f[C]--fast-list\f[R] and 71381 \f[C]--drive-shared-with-me\f[R] (Nick Craig-Wood) 71382 .IP \[bu] 2 71383 Fix duplicate items when using \f[C]--drive-shared-with-me\f[R] (Nick 71384 Craig-Wood) 71385 .IP \[bu] 2 71386 Extend \f[C]--drive-stop-on-upload-limit\f[R] to respond to 71387 \f[C]teamDriveFileLimitExceeded\f[R]. 71388 (harry) 71389 .IP \[bu] 2 71390 Don\[aq]t delete files with multiple parents to avoid data loss (Nick 71391 Craig-Wood) 71392 .IP \[bu] 2 71393 Server side copy docs use default description if empty (Nick Craig-Wood) 71394 .RE 71395 .IP \[bu] 2 71396 Dropbox 71397 .RS 2 71398 .IP \[bu] 2 71399 Make error insufficient space to be fatal (harry) 71400 .IP \[bu] 2 71401 Add info about required redirect url (Elan Ruusam\[:a]e) 71402 .RE 71403 .IP \[bu] 2 71404 Fichier 71405 .RS 2 71406 .IP \[bu] 2 71407 Add support for \f[C]--header-upload\f[R] and 71408 \f[C]--header-download\f[R] (Tim Gallant) 71409 .IP \[bu] 2 71410 Implement custom pacer to deal with the new rate limiting (buengese) 71411 .RE 71412 .IP \[bu] 2 71413 FTP 71414 .RS 2 71415 .IP \[bu] 2 71416 Fix lockup when using concurrency limit on failed connections (Nick 71417 Craig-Wood) 71418 .IP \[bu] 2 71419 Fix lockup on failed upload when using concurrency limit (Nick 71420 Craig-Wood) 71421 .IP \[bu] 2 71422 Fix lockup on Close failures when using concurrency limit (Nick 71423 Craig-Wood) 71424 .IP \[bu] 2 71425 Work around pureftp sending spurious 150 messages (Nick Craig-Wood) 71426 .RE 71427 .IP \[bu] 2 71428 Google Cloud Storage 71429 .RS 2 71430 .IP \[bu] 2 71431 Add support for \f[C]--header-upload\f[R] and 71432 \f[C]--header-download\f[R] (Nick Craig-Wood) 71433 .IP \[bu] 2 71434 Add \f[C]ARCHIVE\f[R] storage class to help (Adam Stroud) 71435 .IP \[bu] 2 71436 Ignore directory markers at the root (Nick Craig-Wood) 71437 .RE 71438 .IP \[bu] 2 71439 Googlephotos 71440 .RS 2 71441 .IP \[bu] 2 71442 Make the start year configurable (Daven) 71443 .IP \[bu] 2 71444 Add support for \f[C]--header-upload\f[R] and 71445 \f[C]--header-download\f[R] (Tim Gallant) 71446 .IP \[bu] 2 71447 Create feature/favorites directory (Brandon Philips) 71448 .IP \[bu] 2 71449 Fix \[dq]concurrent map write\[dq] error (Nick Craig-Wood) 71450 .IP \[bu] 2 71451 Don\[aq]t put an image in error message (Nick Craig-Wood) 71452 .RE 71453 .IP \[bu] 2 71454 HTTP 71455 .RS 2 71456 .IP \[bu] 2 71457 Improved directory listing with new template from Caddy project 71458 (calisro) 71459 .RE 71460 .IP \[bu] 2 71461 Jottacloud 71462 .RS 2 71463 .IP \[bu] 2 71464 Implement \f[C]--jottacloud-trashed-only\f[R] (buengese) 71465 .IP \[bu] 2 71466 Add support for \f[C]--header-upload\f[R] and 71467 \f[C]--header-download\f[R] (Tim Gallant) 71468 .IP \[bu] 2 71469 Use \f[C]RawURLEncoding\f[R] when decoding base64 encoded login token 71470 (buengese) 71471 .IP \[bu] 2 71472 Implement cleanup (buengese) 71473 .IP \[bu] 2 71474 Update docs regarding cleanup, removed remains from old auth, and added 71475 warning about special mountpoints. 71476 (albertony) 71477 .RE 71478 .IP \[bu] 2 71479 Mailru 71480 .RS 2 71481 .IP \[bu] 2 71482 Describe 2FA requirements (valery1707) 71483 .RE 71484 .IP \[bu] 2 71485 Onedrive 71486 .RS 2 71487 .IP \[bu] 2 71488 Implement \f[C]--onedrive-server-side-across-configs\f[R] (Nick 71489 Craig-Wood) 71490 .IP \[bu] 2 71491 Add support for \f[C]--header-upload\f[R] and 71492 \f[C]--header-download\f[R] (Tim Gallant) 71493 .IP \[bu] 2 71494 Fix occasional 416 errors on multipart uploads (Nick Craig-Wood) 71495 .IP \[bu] 2 71496 Added maximum chunk size limit warning in the docs (Harry) 71497 .IP \[bu] 2 71498 Fix missing drive on config (Nick Craig-Wood) 71499 .IP \[bu] 2 71500 Make error \f[C]quotaLimitReached\f[R] to be fatal (harry) 71501 .RE 71502 .IP \[bu] 2 71503 Opendrive 71504 .RS 2 71505 .IP \[bu] 2 71506 Add support for \f[C]--header-upload\f[R] and 71507 \f[C]--header-download\f[R] (Tim Gallant) 71508 .RE 71509 .IP \[bu] 2 71510 Pcloud 71511 .RS 2 71512 .IP \[bu] 2 71513 Added support for interchangeable root folder for pCloud backend (Sunil 71514 Patra) 71515 .IP \[bu] 2 71516 Add support for \f[C]--header-upload\f[R] and 71517 \f[C]--header-download\f[R] (Tim Gallant) 71518 .IP \[bu] 2 71519 Fix initial config \[dq]Auth state doesn\[aq]t match\[dq] message (Nick 71520 Craig-Wood) 71521 .RE 71522 .IP \[bu] 2 71523 Premiumizeme 71524 .RS 2 71525 .IP \[bu] 2 71526 Add support for \f[C]--header-upload\f[R] and 71527 \f[C]--header-download\f[R] (Tim Gallant) 71528 .IP \[bu] 2 71529 Prune unused functions (Lars Lehtonen) 71530 .RE 71531 .IP \[bu] 2 71532 Putio 71533 .RS 2 71534 .IP \[bu] 2 71535 Add support for \f[C]--header-upload\f[R] and 71536 \f[C]--header-download\f[R] (Nick Craig-Wood) 71537 .IP \[bu] 2 71538 Make downloading files use the rclone http Client (Nick Craig-Wood) 71539 .IP \[bu] 2 71540 Fix parsing of remotes with leading and trailing / (Nick Craig-Wood) 71541 .RE 71542 .IP \[bu] 2 71543 Qingstor 71544 .RS 2 71545 .IP \[bu] 2 71546 Make \f[C]rclone cleanup\f[R] remove pending multipart uploads older 71547 than 24h (Nick Craig-Wood) 71548 .IP \[bu] 2 71549 Try harder to cancel failed multipart uploads (Nick Craig-Wood) 71550 .IP \[bu] 2 71551 Prune \f[C]multiUploader.list()\f[R] (Lars Lehtonen) 71552 .IP \[bu] 2 71553 Lint fix (Lars Lehtonen) 71554 .RE 71555 .IP \[bu] 2 71556 S3 71557 .RS 2 71558 .IP \[bu] 2 71559 Add support for \f[C]--header-upload\f[R] and 71560 \f[C]--header-download\f[R] (Tim Gallant) 71561 .IP \[bu] 2 71562 Use memory pool for buffer allocations (Maciej Zimnoch) 71563 .IP \[bu] 2 71564 Add SSE-C support for AWS, Ceph, and MinIO (Jack Anderson) 71565 .IP \[bu] 2 71566 Fail fast multipart upload (Micha\[/l] Matczuk) 71567 .IP \[bu] 2 71568 Report errors on bucket creation (mkdir) correctly (Nick Craig-Wood) 71569 .IP \[bu] 2 71570 Specify that Minio supports URL encoding in listings (Nick Craig-Wood) 71571 .IP \[bu] 2 71572 Added 500 as retryErrorCode (Micha\[/l] Matczuk) 71573 .IP \[bu] 2 71574 Use \f[C]--low-level-retries\f[R] as the number of SDK retries 71575 (Aleksandar Jankovi\['c]) 71576 .IP \[bu] 2 71577 Fix multipart abort context (Aleksandar Jankovic) 71578 .IP \[bu] 2 71579 Replace deprecated \f[C]session.New()\f[R] with 71580 \f[C]session.NewSession()\f[R] (Lars Lehtonen) 71581 .IP \[bu] 2 71582 Use the provided size parameter when allocating a new memory pool 71583 (Joachim Brandon LeBlanc) 71584 .IP \[bu] 2 71585 Use rclone\[aq]s low level retries instead of AWS SDK to fix listing 71586 retries (Nick Craig-Wood) 71587 .IP \[bu] 2 71588 Ignore directory markers at the root also (Nick Craig-Wood) 71589 .IP \[bu] 2 71590 Use single memory pool (Micha\[/l] Matczuk) 71591 .IP \[bu] 2 71592 Do not resize buf on put to memBuf (Micha\[/l] Matczuk) 71593 .IP \[bu] 2 71594 Improve docs for \f[C]--s3-disable-checksum\f[R] (Nick Craig-Wood) 71595 .IP \[bu] 2 71596 Don\[aq]t leak memory or tokens in edge cases for multipart upload (Nick 71597 Craig-Wood) 71598 .RE 71599 .IP \[bu] 2 71600 Seafile 71601 .RS 2 71602 .IP \[bu] 2 71603 Implement 2FA (Fred) 71604 .RE 71605 .IP \[bu] 2 71606 SFTP 71607 .RS 2 71608 .IP \[bu] 2 71609 Added \f[C]--sftp-pem-key\f[R] to support inline key files (calisro) 71610 .IP \[bu] 2 71611 Fix post transfer copies failing with 0 size when using 71612 \f[C]set_modtime=false\f[R] (Nick Craig-Wood) 71613 .RE 71614 .IP \[bu] 2 71615 Sharefile 71616 .RS 2 71617 .IP \[bu] 2 71618 Add support for \f[C]--header-upload\f[R] and 71619 \f[C]--header-download\f[R] (Tim Gallant) 71620 .RE 71621 .IP \[bu] 2 71622 Sugarsync 71623 .RS 2 71624 .IP \[bu] 2 71625 Add support for \f[C]--header-upload\f[R] and 71626 \f[C]--header-download\f[R] (Tim Gallant) 71627 .RE 71628 .IP \[bu] 2 71629 Swift 71630 .RS 2 71631 .IP \[bu] 2 71632 Add support for \f[C]--header-upload\f[R] and 71633 \f[C]--header-download\f[R] (Nick Craig-Wood) 71634 .IP \[bu] 2 71635 Fix cosmetic issue in error message (Martin Michlmayr) 71636 .RE 71637 .IP \[bu] 2 71638 Union 71639 .RS 2 71640 .IP \[bu] 2 71641 Implement multiple writable remotes (Max Sum) 71642 .IP \[bu] 2 71643 Fix server-side copy (Max Sum) 71644 .IP \[bu] 2 71645 Implement ListR (Max Sum) 71646 .IP \[bu] 2 71647 Enable ListR when upstreams contain local (Max Sum) 71648 .RE 71649 .IP \[bu] 2 71650 WebDAV 71651 .RS 2 71652 .IP \[bu] 2 71653 Add support for \f[C]--header-upload\f[R] and 71654 \f[C]--header-download\f[R] (Tim Gallant) 71655 .IP \[bu] 2 71656 Fix \f[C]X-OC-Mtime\f[R] header for Transip compatibility (Nick 71657 Craig-Wood) 71658 .IP \[bu] 2 71659 Report full and consistent usage with \f[C]about\f[R] (Yves G) 71660 .RE 71661 .IP \[bu] 2 71662 Yandex 71663 .RS 2 71664 .IP \[bu] 2 71665 Add support for \f[C]--header-upload\f[R] and 71666 \f[C]--header-download\f[R] (Tim Gallant) 71667 .RE 71668 .SS v1.51.0 - 2020-02-01 71669 .IP \[bu] 2 71670 New backends 71671 .RS 2 71672 .IP \[bu] 2 71673 Memory (https://rclone.org/memory/) (Nick Craig-Wood) 71674 .IP \[bu] 2 71675 Sugarsync (https://rclone.org/sugarsync/) (Nick Craig-Wood) 71676 .RE 71677 .IP \[bu] 2 71678 New Features 71679 .RS 2 71680 .IP \[bu] 2 71681 Adjust all backends to have \f[C]--backend-encoding\f[R] parameter (Nick 71682 Craig-Wood) 71683 .RS 2 71684 .IP \[bu] 2 71685 this enables the encoding for special characters to be adjusted or 71686 disabled 71687 .RE 71688 .IP \[bu] 2 71689 Add \f[C]--max-duration\f[R] flag to control the maximum duration of a 71690 transfer session (boosh) 71691 .IP \[bu] 2 71692 Add \f[C]--expect-continue-timeout\f[R] flag, default 1s (Nick 71693 Craig-Wood) 71694 .IP \[bu] 2 71695 Add \f[C]--no-check-dest\f[R] flag for copying without testing the 71696 destination (Nick Craig-Wood) 71697 .IP \[bu] 2 71698 Implement \f[C]--order-by\f[R] flag to order transfers (Nick Craig-Wood) 71699 .IP \[bu] 2 71700 accounting 71701 .RS 2 71702 .IP \[bu] 2 71703 Don\[aq]t show entries in both transferring and checking (Nick 71704 Craig-Wood) 71705 .IP \[bu] 2 71706 Add option to delete stats (Aleksandar Jankovic) 71707 .RE 71708 .IP \[bu] 2 71709 build 71710 .RS 2 71711 .IP \[bu] 2 71712 Compress the test builds with gzip (Nick Craig-Wood) 71713 .IP \[bu] 2 71714 Implement a framework for starting test servers during tests (Nick 71715 Craig-Wood) 71716 .RE 71717 .IP \[bu] 2 71718 cmd: Always print elapsed time to tenth place seconds in progress (Gary 71719 Kim) 71720 .IP \[bu] 2 71721 config 71722 .RS 2 71723 .IP \[bu] 2 71724 Add \f[C]--password-command\f[R] to allow dynamic config password (Damon 71725 Permezel) 71726 .IP \[bu] 2 71727 Give config questions default values (Nick Craig-Wood) 71728 .IP \[bu] 2 71729 Check a remote exists when creating a new one (Nick Craig-Wood) 71730 .RE 71731 .IP \[bu] 2 71732 copyurl: Add \f[C]--stdout\f[R] flag to write to stdout (Nick 71733 Craig-Wood) 71734 .IP \[bu] 2 71735 dedupe: Implement keep smallest too (Nick Craig-Wood) 71736 .IP \[bu] 2 71737 hashsum: Add flag \f[C]--base64\f[R] flag (landall) 71738 .IP \[bu] 2 71739 lsf: Speed up on s3/swift/etc by not reading mimetype by default (Nick 71740 Craig-Wood) 71741 .IP \[bu] 2 71742 lsjson: Add \f[C]--no-mimetype\f[R] flag (Nick Craig-Wood) 71743 .IP \[bu] 2 71744 rc: Add methods to turn on blocking and mutex profiling (Nick 71745 Craig-Wood) 71746 .IP \[bu] 2 71747 rcd 71748 .RS 2 71749 .IP \[bu] 2 71750 Adding group parameter to stats (Chaitanya) 71751 .IP \[bu] 2 71752 Move webgui apart; option to disable browser (Xiaoxing Ye) 71753 .RE 71754 .IP \[bu] 2 71755 serve sftp: Add support for public key with auth proxy (Paul Tinsley) 71756 .IP \[bu] 2 71757 stats: Show deletes in stats and hide zero stats (anuar45) 71758 .RE 71759 .IP \[bu] 2 71760 Bug Fixes 71761 .RS 2 71762 .IP \[bu] 2 71763 accounting 71764 .RS 2 71765 .IP \[bu] 2 71766 Fix error counter counting multiple times (Ankur Gupta) 71767 .IP \[bu] 2 71768 Fix error count shown as checks (Cnly) 71769 .IP \[bu] 2 71770 Clear finished transfer in stats-reset (Maciej Zimnoch) 71771 .IP \[bu] 2 71772 Added StatsInfo locking in statsGroups sum function (Micha\[/l] Matczuk) 71773 .RE 71774 .IP \[bu] 2 71775 asyncreader: Fix EOF error (buengese) 71776 .IP \[bu] 2 71777 check: Fix \f[C]--one-way\f[R] recursing more directories than it needs 71778 to (Nick Craig-Wood) 71779 .IP \[bu] 2 71780 chunkedreader: Disable hash calculation for first segment (Nick 71781 Craig-Wood) 71782 .IP \[bu] 2 71783 config 71784 .RS 2 71785 .IP \[bu] 2 71786 Do not open browser on headless on drive/gcs/google photos (Xiaoxing Ye) 71787 .IP \[bu] 2 71788 SetValueAndSave ignore error if config section does not exist yet 71789 (buengese) 71790 .RE 71791 .IP \[bu] 2 71792 cmd: Fix completion with an encrypted config (Danil Semelenov) 71793 .IP \[bu] 2 71794 dbhashsum: Stop it returning UNSUPPORTED on dropbox (Nick Craig-Wood) 71795 .IP \[bu] 2 71796 dedupe: Add missing modes to help string (Nick Craig-Wood) 71797 .IP \[bu] 2 71798 operations 71799 .RS 2 71800 .IP \[bu] 2 71801 Fix dedupe continuing on errors like insufficientFilePersimmon 71802 (SezalAgrawal) 71803 .IP \[bu] 2 71804 Clear accounting before low level retry (Maciej Zimnoch) 71805 .IP \[bu] 2 71806 Write debug message when hashes could not be checked (Ole Sch\[:u]tt) 71807 .IP \[bu] 2 71808 Move interface assertion to tests to remove pflag dependency (Nick 71809 Craig-Wood) 71810 .IP \[bu] 2 71811 Make NewOverrideObjectInfo public and factor uses (Nick Craig-Wood) 71812 .RE 71813 .IP \[bu] 2 71814 proxy: Replace use of bcrypt with sha256 (Nick Craig-Wood) 71815 .IP \[bu] 2 71816 vendor 71817 .RS 2 71818 .IP \[bu] 2 71819 Update bazil.org/fuse to fix FreeBSD 12.1 (Nick Craig-Wood) 71820 .IP \[bu] 2 71821 Update github.com/t3rm1n4l/go-mega to fix mega \[dq]illegal base64 data 71822 at input byte 22\[dq] (Nick Craig-Wood) 71823 .IP \[bu] 2 71824 Update termbox-go to fix ncdu command on FreeBSD (Kuang-che Wu) 71825 .IP \[bu] 2 71826 Update t3rm1n4l/go-mega - fixes mega: couldn\[aq]t login: crypto/aes: 71827 invalid key size 0 (Nick Craig-Wood) 71828 .RE 71829 .RE 71830 .IP \[bu] 2 71831 Mount 71832 .RS 2 71833 .IP \[bu] 2 71834 Enable async reads for a 20% speedup (Nick Craig-Wood) 71835 .IP \[bu] 2 71836 Replace use of WriteAt with Write for cache mode >= writes and O_APPEND 71837 (Brett Dutro) 71838 .IP \[bu] 2 71839 Make sure we call unmount when exiting (Nick Craig-Wood) 71840 .IP \[bu] 2 71841 Don\[aq]t build on go1.10 as bazil/fuse no longer supports it (Nick 71842 Craig-Wood) 71843 .IP \[bu] 2 71844 When setting dates discard out of range dates (Nick Craig-Wood) 71845 .RE 71846 .IP \[bu] 2 71847 VFS 71848 .RS 2 71849 .IP \[bu] 2 71850 Add a newly created file straight into the directory (Nick Craig-Wood) 71851 .IP \[bu] 2 71852 Only calculate one hash for reads for a speedup (Nick Craig-Wood) 71853 .IP \[bu] 2 71854 Make ReadAt for non cached files work better with non-sequential reads 71855 (Nick Craig-Wood) 71856 .IP \[bu] 2 71857 Fix edge cases when reading ModTime from file (Nick Craig-Wood) 71858 .IP \[bu] 2 71859 Make sure existing files opened for write show correct size (Nick 71860 Craig-Wood) 71861 .IP \[bu] 2 71862 Don\[aq]t cache the path in RW file objects to fix renaming (Nick 71863 Craig-Wood) 71864 .IP \[bu] 2 71865 Fix rename of open files when using the VFS cache (Nick Craig-Wood) 71866 .IP \[bu] 2 71867 When renaming files in the cache, rename the cache item in memory too 71868 (Nick Craig-Wood) 71869 .IP \[bu] 2 71870 Fix open file renaming on drive when using 71871 \f[C]--vfs-cache-mode writes\f[R] (Nick Craig-Wood) 71872 .IP \[bu] 2 71873 Fix incorrect modtime for mv into mount with 71874 \f[C]--vfs-cache-modes writes\f[R] (Nick Craig-Wood) 71875 .IP \[bu] 2 71876 On rename, rename in cache too if the file exists (Anagh Kumar Baranwal) 71877 .RE 71878 .IP \[bu] 2 71879 Local 71880 .RS 2 71881 .IP \[bu] 2 71882 Make source file being updated errors be NoLowLevelRetry errors (Nick 71883 Craig-Wood) 71884 .IP \[bu] 2 71885 Fix update of hidden files on Windows (Nick Craig-Wood) 71886 .RE 71887 .IP \[bu] 2 71888 Cache 71889 .RS 2 71890 .IP \[bu] 2 71891 Follow move of upstream library github.com/coreos/bbolt 71892 github.com/etcd-io/bbolt (Nick Craig-Wood) 71893 .IP \[bu] 2 71894 Fix \f[C]fatal error: concurrent map writes\f[R] (Nick Craig-Wood) 71895 .RE 71896 .IP \[bu] 2 71897 Crypt 71898 .RS 2 71899 .IP \[bu] 2 71900 Reorder the filename encryption options (Thomas Eales) 71901 .IP \[bu] 2 71902 Correctly handle trailing dot (buengese) 71903 .RE 71904 .IP \[bu] 2 71905 Chunker 71906 .RS 2 71907 .IP \[bu] 2 71908 Reduce length of temporary suffix (Ivan Andreev) 71909 .RE 71910 .IP \[bu] 2 71911 Drive 71912 .RS 2 71913 .IP \[bu] 2 71914 Add \f[C]--drive-stop-on-upload-limit\f[R] flag to stop syncs when 71915 upload limit reached (Nick Craig-Wood) 71916 .IP \[bu] 2 71917 Add \f[C]--drive-use-shared-date\f[R] to use date file was shared 71918 instead of modified date (Garry McNulty) 71919 .IP \[bu] 2 71920 Make sure invalid auth for teamdrives always reports an error (Nick 71921 Craig-Wood) 71922 .IP \[bu] 2 71923 Fix \f[C]--fast-list\f[R] when using appDataFolder (Nick Craig-Wood) 71924 .IP \[bu] 2 71925 Use multipart resumable uploads for streaming and uploads in mount (Nick 71926 Craig-Wood) 71927 .IP \[bu] 2 71928 Log an ERROR if an incomplete search is returned (Nick Craig-Wood) 71929 .IP \[bu] 2 71930 Hide dangerous config from the configurator (Nick Craig-Wood) 71931 .RE 71932 .IP \[bu] 2 71933 Dropbox 71934 .RS 2 71935 .IP \[bu] 2 71936 Treat \f[C]insufficient_space\f[R] errors as non retriable errors (Nick 71937 Craig-Wood) 71938 .RE 71939 .IP \[bu] 2 71940 Jottacloud 71941 .RS 2 71942 .IP \[bu] 2 71943 Use new auth method used by official client (buengese) 71944 .IP \[bu] 2 71945 Add URL to generate Login Token to config wizard (Nick Craig-Wood) 71946 .IP \[bu] 2 71947 Add support whitelabel versions (buengese) 71948 .RE 71949 .IP \[bu] 2 71950 Koofr 71951 .RS 2 71952 .IP \[bu] 2 71953 Use rclone HTTP client. 71954 (jaKa) 71955 .RE 71956 .IP \[bu] 2 71957 Onedrive 71958 .RS 2 71959 .IP \[bu] 2 71960 Add Sites.Read.All permission (Benjamin Richter) 71961 .IP \[bu] 2 71962 Add support \[dq]Retry-After\[dq] header (Motonori IWAMURO) 71963 .RE 71964 .IP \[bu] 2 71965 Opendrive 71966 .RS 2 71967 .IP \[bu] 2 71968 Implement \f[C]--opendrive-chunk-size\f[R] (Nick Craig-Wood) 71969 .RE 71970 .IP \[bu] 2 71971 S3 71972 .RS 2 71973 .IP \[bu] 2 71974 Re-implement multipart upload to fix memory issues (Nick Craig-Wood) 71975 .IP \[bu] 2 71976 Add \f[C]--s3-copy-cutoff\f[R] for size to switch to multipart copy 71977 (Nick Craig-Wood) 71978 .IP \[bu] 2 71979 Add new region Asia Pacific (Hong Kong) (Outvi V) 71980 .IP \[bu] 2 71981 Reduce memory usage streaming files by reducing max stream upload size 71982 (Nick Craig-Wood) 71983 .IP \[bu] 2 71984 Add \f[C]--s3-list-chunk\f[R] option for bucket listing (Thomas 71985 Kriechbaumer) 71986 .IP \[bu] 2 71987 Force path style bucket access to off for AWS deprecation (Nick 71988 Craig-Wood) 71989 .IP \[bu] 2 71990 Use AWS web identity role provider if available (Tennix) 71991 .IP \[bu] 2 71992 Add StackPath Object Storage Support (Dave Koston) 71993 .IP \[bu] 2 71994 Fix ExpiryWindow value (Aleksandar Jankovic) 71995 .IP \[bu] 2 71996 Fix DisableChecksum condition (Aleksandar Jankovi\['c]) 71997 .IP \[bu] 2 71998 Fix URL decoding of NextMarker (Nick Craig-Wood) 71999 .RE 72000 .IP \[bu] 2 72001 SFTP 72002 .RS 2 72003 .IP \[bu] 2 72004 Add \f[C]--sftp-skip-links\f[R] to skip symlinks and non regular files 72005 (Nick Craig-Wood) 72006 .IP \[bu] 2 72007 Retry Creation of Connection (Sebastian Brandt) 72008 .IP \[bu] 2 72009 Fix \[dq]failed to parse private key file: ssh: not an encrypted 72010 key\[dq] error (Nick Craig-Wood) 72011 .IP \[bu] 2 72012 Open files for update write only to fix AWS SFTP interop (Nick 72013 Craig-Wood) 72014 .RE 72015 .IP \[bu] 2 72016 Swift 72017 .RS 2 72018 .IP \[bu] 2 72019 Reserve segments of dynamic large object when delete objects in 72020 container what was enabled versioning. 72021 (Nguy\[u1EC5]n H\[u1EEF]u Lu\[^a]n) 72022 .IP \[bu] 2 72023 Fix parsing of X-Object-Manifest (Nick Craig-Wood) 72024 .IP \[bu] 2 72025 Update OVH API endpoint (unbelauscht) 72026 .RE 72027 .IP \[bu] 2 72028 WebDAV 72029 .RS 2 72030 .IP \[bu] 2 72031 Make nextcloud only upload SHA1 checksums (Nick Craig-Wood) 72032 .IP \[bu] 2 72033 Fix case of \[dq]Bearer\[dq] in Authorization: header to agree with RFC 72034 (Nick Craig-Wood) 72035 .IP \[bu] 2 72036 Add Referer header to fix problems with WAFs (Nick Craig-Wood) 72037 .RE 72038 .SS v1.50.2 - 2019-11-19 72039 .IP \[bu] 2 72040 Bug Fixes 72041 .RS 2 72042 .IP \[bu] 2 72043 accounting: Fix memory leak on retries operations (Nick Craig-Wood) 72044 .RE 72045 .IP \[bu] 2 72046 Drive 72047 .RS 2 72048 .IP \[bu] 2 72049 Fix listing of the root directory with drive.files scope (Nick 72050 Craig-Wood) 72051 .IP \[bu] 2 72052 Fix --drive-root-folder-id with team/shared drives (Nick Craig-Wood) 72053 .RE 72054 .SS v1.50.1 - 2019-11-02 72055 .IP \[bu] 2 72056 Bug Fixes 72057 .RS 2 72058 .IP \[bu] 2 72059 hash: Fix accidentally changed hash names for \f[C]DropboxHash\f[R] and 72060 \f[C]CRC-32\f[R] (Nick Craig-Wood) 72061 .IP \[bu] 2 72062 fshttp: Fix error reporting on tpslimit token bucket errors (Nick 72063 Craig-Wood) 72064 .IP \[bu] 2 72065 fshttp: Don\[aq]t print token bucket errors on context cancelled (Nick 72066 Craig-Wood) 72067 .RE 72068 .IP \[bu] 2 72069 Local 72070 .RS 2 72071 .IP \[bu] 2 72072 Fix listings of . 72073 on Windows (Nick Craig-Wood) 72074 .RE 72075 .IP \[bu] 2 72076 Onedrive 72077 .RS 2 72078 .IP \[bu] 2 72079 Fix DirMove/Move after Onedrive change (Xiaoxing Ye) 72080 .RE 72081 .SS v1.50.0 - 2019-10-26 72082 .IP \[bu] 2 72083 New backends 72084 .RS 2 72085 .IP \[bu] 2 72086 Citrix Sharefile (https://rclone.org/sharefile/) (Nick Craig-Wood) 72087 .IP \[bu] 2 72088 Chunker (https://rclone.org/chunker/) - an overlay backend to split 72089 files into smaller parts (Ivan Andreev) 72090 .IP \[bu] 2 72091 Mail.ru Cloud (https://rclone.org/mailru/) (Ivan Andreev) 72092 .RE 72093 .IP \[bu] 2 72094 New Features 72095 .RS 2 72096 .IP \[bu] 2 72097 encodings (Fabian M\[:o]ller & Nick Craig-Wood) 72098 .RS 2 72099 .IP \[bu] 2 72100 All backends now use file name encoding to ensure any file name can be 72101 written to any backend. 72102 .IP \[bu] 2 72103 See the restricted file name 72104 docs (https://rclone.org/overview/#restricted-filenames) for more info 72105 and the local backend docs. 72106 .IP \[bu] 2 72107 Some file names may look different in rclone if you are using any 72108 control characters in names or unicode FULLWIDTH 72109 symbols (https://en.wikipedia.org/wiki/Halfwidth_and_Fullwidth_Forms_(Unicode_block)). 72110 .RE 72111 .IP \[bu] 2 72112 build 72113 .RS 2 72114 .IP \[bu] 2 72115 Update to use go1.13 for the build (Nick Craig-Wood) 72116 .IP \[bu] 2 72117 Drop support for go1.9 (Nick Craig-Wood) 72118 .IP \[bu] 2 72119 Build rclone with GitHub actions (Nick Craig-Wood) 72120 .IP \[bu] 2 72121 Convert python scripts to python3 (Nick Craig-Wood) 72122 .IP \[bu] 2 72123 Swap Azure/go-ansiterm for mattn/go-colorable (Nick Craig-Wood) 72124 .IP \[bu] 2 72125 Dockerfile fixes (Matei David) 72126 .IP \[bu] 2 72127 Add plugin 72128 support (https://github.com/artpar/artpar/blob/master/CONTRIBUTING.md#writing-a-plugin) 72129 for backends and commands (Richard Patel) 72130 .RE 72131 .IP \[bu] 2 72132 config 72133 .RS 2 72134 .IP \[bu] 2 72135 Use alternating Red/Green in config to make more obvious (Nick 72136 Craig-Wood) 72137 .RE 72138 .IP \[bu] 2 72139 contrib 72140 .RS 2 72141 .IP \[bu] 2 72142 Add sample DLNA server Docker Compose manifest. 72143 (pataquets) 72144 .IP \[bu] 2 72145 Add sample WebDAV server Docker Compose manifest. 72146 (pataquets) 72147 .RE 72148 .IP \[bu] 2 72149 copyurl 72150 .RS 2 72151 .IP \[bu] 2 72152 Add \f[C]--auto-filename\f[R] flag for using file name from URL in 72153 destination path (Denis) 72154 .RE 72155 .IP \[bu] 2 72156 serve dlna: 72157 .RS 2 72158 .IP \[bu] 2 72159 Many compatibility improvements (Dan Walters) 72160 .IP \[bu] 2 72161 Support for external srt subtitles (Dan Walters) 72162 .RE 72163 .IP \[bu] 2 72164 rc 72165 .RS 2 72166 .IP \[bu] 2 72167 Added command core/quit (Saksham Khanna) 72168 .RE 72169 .RE 72170 .IP \[bu] 2 72171 Bug Fixes 72172 .RS 2 72173 .IP \[bu] 2 72174 sync 72175 .RS 2 72176 .IP \[bu] 2 72177 Make \f[C]--update\f[R]/\f[C]-u\f[R] not transfer files that haven\[aq]t 72178 changed (Nick Craig-Wood) 72179 .IP \[bu] 2 72180 Free objects after they come out of the transfer pipe to save memory 72181 (Nick Craig-Wood) 72182 .IP \[bu] 2 72183 Fix \f[C]--files-from without --no-traverse\f[R] doing a recursive scan 72184 (Nick Craig-Wood) 72185 .RE 72186 .IP \[bu] 2 72187 operations 72188 .RS 2 72189 .IP \[bu] 2 72190 Fix accounting for server-side copies (Nick Craig-Wood) 72191 .IP \[bu] 2 72192 Display \[aq]All duplicates removed\[aq] only if dedupe successful 72193 (Sezal Agrawal) 72194 .IP \[bu] 2 72195 Display \[aq]Deleted X extra copies\[aq] only if dedupe successful 72196 (Sezal Agrawal) 72197 .RE 72198 .IP \[bu] 2 72199 accounting 72200 .RS 2 72201 .IP \[bu] 2 72202 Only allow up to 100 completed transfers in the accounting list to save 72203 memory (Nick Craig-Wood) 72204 .IP \[bu] 2 72205 Cull the old time ranges when possible to save memory (Nick Craig-Wood) 72206 .IP \[bu] 2 72207 Fix panic due to server-side copy fallback (Ivan Andreev) 72208 .IP \[bu] 2 72209 Fix memory leak noticeable for transfers of large numbers of objects 72210 (Nick Craig-Wood) 72211 .IP \[bu] 2 72212 Fix total duration calculation (Nick Craig-Wood) 72213 .RE 72214 .IP \[bu] 2 72215 cmd 72216 .RS 2 72217 .IP \[bu] 2 72218 Fix environment variables not setting command line flags (Nick 72219 Craig-Wood) 72220 .IP \[bu] 2 72221 Make autocomplete compatible with bash\[aq]s posix mode for macOS (Danil 72222 Semelenov) 72223 .IP \[bu] 2 72224 Make \f[C]--progress\f[R] work in git bash on Windows (Nick Craig-Wood) 72225 .IP \[bu] 2 72226 Fix \[aq]compopt: command not found\[aq] on autocomplete on macOS (Danil 72227 Semelenov) 72228 .RE 72229 .IP \[bu] 2 72230 config 72231 .RS 2 72232 .IP \[bu] 2 72233 Fix setting of non top level flags from environment variables (Nick 72234 Craig-Wood) 72235 .IP \[bu] 2 72236 Check config names more carefully and report errors (Nick Craig-Wood) 72237 .IP \[bu] 2 72238 Remove error: can\[aq]t use \f[C]--size-only\f[R] and 72239 \f[C]--ignore-size\f[R] together. 72240 (Nick Craig-Wood) 72241 .RE 72242 .IP \[bu] 2 72243 filter: Prevent mixing options when \f[C]--files-from\f[R] is in use 72244 (Michele Caci) 72245 .IP \[bu] 2 72246 serve sftp: Fix crash on unsupported operations (e.g. 72247 Readlink) (Nick Craig-Wood) 72248 .RE 72249 .IP \[bu] 2 72250 Mount 72251 .RS 2 72252 .IP \[bu] 2 72253 Allow files of unknown size to be read properly (Nick Craig-Wood) 72254 .IP \[bu] 2 72255 Skip tests on <= 2 CPUs to avoid lockup (Nick Craig-Wood) 72256 .IP \[bu] 2 72257 Fix panic on File.Open (Nick Craig-Wood) 72258 .IP \[bu] 2 72259 Fix \[dq]mount_fusefs: -o timeout=: option not supported\[dq] on FreeBSD 72260 (Nick Craig-Wood) 72261 .IP \[bu] 2 72262 Don\[aq]t pass huge filenames (>4k) to FUSE as it can\[aq]t cope (Nick 72263 Craig-Wood) 72264 .RE 72265 .IP \[bu] 2 72266 VFS 72267 .RS 2 72268 .IP \[bu] 2 72269 Add flag \f[C]--vfs-case-insensitive\f[R] for windows/macOS mounts (Ivan 72270 Andreev) 72271 .IP \[bu] 2 72272 Make objects of unknown size readable through the VFS (Nick Craig-Wood) 72273 .IP \[bu] 2 72274 Move writeback of dirty data out of close() method into its own method 72275 (FlushWrites) and remove close() call from Flush() (Brett Dutro) 72276 .IP \[bu] 2 72277 Stop empty dirs disappearing when renamed on bucket-based remotes (Nick 72278 Craig-Wood) 72279 .IP \[bu] 2 72280 Stop change notify polling clearing so much of the directory cache (Nick 72281 Craig-Wood) 72282 .RE 72283 .IP \[bu] 2 72284 Azure Blob 72285 .RS 2 72286 .IP \[bu] 2 72287 Disable logging to the Windows event log (Nick Craig-Wood) 72288 .RE 72289 .IP \[bu] 2 72290 B2 72291 .RS 2 72292 .IP \[bu] 2 72293 Remove \f[C]unverified:\f[R] prefix on sha1 to improve interop (e.g. 72294 with CyberDuck) (Nick Craig-Wood) 72295 .RE 72296 .IP \[bu] 2 72297 Box 72298 .RS 2 72299 .IP \[bu] 2 72300 Add options to get access token via JWT auth (David) 72301 .RE 72302 .IP \[bu] 2 72303 Drive 72304 .RS 2 72305 .IP \[bu] 2 72306 Disable HTTP/2 by default to work around INTERNAL_ERROR problems (Nick 72307 Craig-Wood) 72308 .IP \[bu] 2 72309 Make sure that drive root ID is always canonical (Nick Craig-Wood) 72310 .IP \[bu] 2 72311 Fix \f[C]--drive-shared-with-me\f[R] from the root with lsand 72312 \f[C]--fast-list\f[R] (Nick Craig-Wood) 72313 .IP \[bu] 2 72314 Fix ChangeNotify polling for shared drives (Nick Craig-Wood) 72315 .IP \[bu] 2 72316 Fix change notify polling when using appDataFolder (Nick Craig-Wood) 72317 .RE 72318 .IP \[bu] 2 72319 Dropbox 72320 .RS 2 72321 .IP \[bu] 2 72322 Make disallowed filenames errors not retry (Nick Craig-Wood) 72323 .IP \[bu] 2 72324 Fix nil pointer exception on restricted files (Nick Craig-Wood) 72325 .RE 72326 .IP \[bu] 2 72327 Fichier 72328 .RS 2 72329 .IP \[bu] 2 72330 Fix accessing files > 2GB on 32 bit systems (Nick Craig-Wood) 72331 .RE 72332 .IP \[bu] 2 72333 FTP 72334 .RS 2 72335 .IP \[bu] 2 72336 Allow disabling EPSV mode (Jon Fautley) 72337 .RE 72338 .IP \[bu] 2 72339 HTTP 72340 .RS 2 72341 .IP \[bu] 2 72342 HEAD directory entries in parallel to speedup (Nick Craig-Wood) 72343 .IP \[bu] 2 72344 Add \f[C]--http-no-head\f[R] to stop rclone doing HEAD in listings (Nick 72345 Craig-Wood) 72346 .RE 72347 .IP \[bu] 2 72348 Putio 72349 .RS 2 72350 .IP \[bu] 2 72351 Add ability to resume uploads (Cenk Alti) 72352 .RE 72353 .IP \[bu] 2 72354 S3 72355 .RS 2 72356 .IP \[bu] 2 72357 Fix signature v2_auth headers (Anthony Rusdi) 72358 .IP \[bu] 2 72359 Fix encoding for control characters (Nick Craig-Wood) 72360 .IP \[bu] 2 72361 Only ask for URL encoded directory listings if we need them on Ceph 72362 (Nick Craig-Wood) 72363 .IP \[bu] 2 72364 Add option for multipart failure behaviour (Aleksandar Jankovic) 72365 .IP \[bu] 2 72366 Support for multipart copy (\[u5E84]\[u5929]\[u7FFC]) 72367 .IP \[bu] 2 72368 Fix nil pointer reference if no metadata returned for object (Nick 72369 Craig-Wood) 72370 .RE 72371 .IP \[bu] 2 72372 SFTP 72373 .RS 2 72374 .IP \[bu] 2 72375 Fix \f[C]--sftp-ask-password\f[R] trying to contact the ssh agent (Nick 72376 Craig-Wood) 72377 .IP \[bu] 2 72378 Fix hashes of files with backslashes (Nick Craig-Wood) 72379 .IP \[bu] 2 72380 Include more ciphers with \f[C]--sftp-use-insecure-cipher\f[R] (Carlos 72381 Ferreyra) 72382 .RE 72383 .IP \[bu] 2 72384 WebDAV 72385 .RS 2 72386 .IP \[bu] 2 72387 Parse and return Sharepoint error response (Henning Surmeier) 72388 .RE 72389 .SS v1.49.5 - 2019-10-05 72390 .IP \[bu] 2 72391 Bug Fixes 72392 .RS 2 72393 .IP \[bu] 2 72394 Revert back to go1.12.x for the v1.49.x builds as go1.13.x was causing 72395 issues (Nick Craig-Wood) 72396 .IP \[bu] 2 72397 Fix rpm packages by using master builds of nfpm (Nick Craig-Wood) 72398 .IP \[bu] 2 72399 Fix macOS build after brew changes (Nick Craig-Wood) 72400 .RE 72401 .SS v1.49.4 - 2019-09-29 72402 .IP \[bu] 2 72403 Bug Fixes 72404 .RS 2 72405 .IP \[bu] 2 72406 cmd/rcd: Address ZipSlip vulnerability (Richard Patel) 72407 .IP \[bu] 2 72408 accounting: Fix file handle leak on errors (Nick Craig-Wood) 72409 .IP \[bu] 2 72410 oauthutil: Fix security problem when running with two users on the same 72411 machine (Nick Craig-Wood) 72412 .RE 72413 .IP \[bu] 2 72414 FTP 72415 .RS 2 72416 .IP \[bu] 2 72417 Fix listing of an empty root returning: error dir not found (Nick 72418 Craig-Wood) 72419 .RE 72420 .IP \[bu] 2 72421 S3 72422 .RS 2 72423 .IP \[bu] 2 72424 Fix SetModTime on GLACIER/ARCHIVE objects and implement set/get tier 72425 (Nick Craig-Wood) 72426 .RE 72427 .SS v1.49.3 - 2019-09-15 72428 .IP \[bu] 2 72429 Bug Fixes 72430 .RS 2 72431 .IP \[bu] 2 72432 accounting 72433 .RS 2 72434 .IP \[bu] 2 72435 Fix total duration calculation (Aleksandar Jankovic) 72436 .IP \[bu] 2 72437 Fix \[dq]file already closed\[dq] on transfer retries (Nick Craig-Wood) 72438 .RE 72439 .RE 72440 .SS v1.49.2 - 2019-09-08 72441 .IP \[bu] 2 72442 New Features 72443 .RS 2 72444 .IP \[bu] 2 72445 build: Add Docker workflow support (Alfonso Montero) 72446 .RE 72447 .IP \[bu] 2 72448 Bug Fixes 72449 .RS 2 72450 .IP \[bu] 2 72451 accounting: Fix locking in Transfer to avoid deadlock with 72452 \f[C]--progress\f[R] (Nick Craig-Wood) 72453 .IP \[bu] 2 72454 docs: Fix template argument for mktemp in install.sh (Cnly) 72455 .IP \[bu] 2 72456 operations: Fix \f[C]-u\f[R]/\f[C]--update\f[R] with google photos / 72457 files of unknown size (Nick Craig-Wood) 72458 .IP \[bu] 2 72459 rc: Fix docs for config/create /update /password (Nick Craig-Wood) 72460 .RE 72461 .IP \[bu] 2 72462 Google Cloud Storage 72463 .RS 2 72464 .IP \[bu] 2 72465 Fix need for elevated permissions on SetModTime (Nick Craig-Wood) 72466 .RE 72467 .SS v1.49.1 - 2019-08-28 72468 .IP \[bu] 2 72469 Bug Fixes 72470 .RS 2 72471 .IP \[bu] 2 72472 config: Fix generated passwords being stored as empty password (Nick 72473 Craig-Wood) 72474 .IP \[bu] 2 72475 rcd: Added missing parameter for web-gui info logs. 72476 (Chaitanya) 72477 .RE 72478 .IP \[bu] 2 72479 Googlephotos 72480 .RS 2 72481 .IP \[bu] 2 72482 Fix crash on error response (Nick Craig-Wood) 72483 .RE 72484 .IP \[bu] 2 72485 Onedrive 72486 .RS 2 72487 .IP \[bu] 2 72488 Fix crash on error response (Nick Craig-Wood) 72489 .RE 72490 .SS v1.49.0 - 2019-08-26 72491 .IP \[bu] 2 72492 New backends 72493 .RS 2 72494 .IP \[bu] 2 72495 1fichier (https://rclone.org/fichier/) (Laura Hausmann) 72496 .IP \[bu] 2 72497 Google Photos (https://rclone.org/googlephotos/) (Nick Craig-Wood) 72498 .IP \[bu] 2 72499 Putio (https://rclone.org/putio/) (Cenk Alti) 72500 .IP \[bu] 2 72501 premiumize.me (https://rclone.org/premiumizeme/) (Nick Craig-Wood) 72502 .RE 72503 .IP \[bu] 2 72504 New Features 72505 .RS 2 72506 .IP \[bu] 2 72507 Experimental web GUI (https://rclone.org/gui/) (Chaitanya Bankanhal) 72508 .IP \[bu] 2 72509 Implement \f[C]--compare-dest\f[R] & \f[C]--copy-dest\f[R] (yparitcher) 72510 .IP \[bu] 2 72511 Implement \f[C]--suffix\f[R] without \f[C]--backup-dir\f[R] for backup 72512 to current dir (yparitcher) 72513 .IP \[bu] 2 72514 \f[C]config reconnect\f[R] to re-login (re-run the oauth login) for the 72515 backend. 72516 (Nick Craig-Wood) 72517 .IP \[bu] 2 72518 \f[C]config userinfo\f[R] to discover which user you are logged in as. 72519 (Nick Craig-Wood) 72520 .IP \[bu] 2 72521 \f[C]config disconnect\f[R] to disconnect you (log out) from the 72522 backend. 72523 (Nick Craig-Wood) 72524 .IP \[bu] 2 72525 Add \f[C]--use-json-log\f[R] for JSON logging (justinalin) 72526 .IP \[bu] 2 72527 Add context propagation to rclone (Aleksandar Jankovic) 72528 .IP \[bu] 2 72529 Reworking internal statistics interfaces so they work with rc jobs 72530 (Aleksandar Jankovic) 72531 .IP \[bu] 2 72532 Add Higher units for ETA (AbelThar) 72533 .IP \[bu] 2 72534 Update rclone logos to new design (Andreas Chlupka) 72535 .IP \[bu] 2 72536 hash: Add CRC-32 support (Cenk Alti) 72537 .IP \[bu] 2 72538 help showbackend: Fixed advanced option category when there are no 72539 standard options (buengese) 72540 .IP \[bu] 2 72541 ncdu: Display/Copy to Clipboard Current Path (Gary Kim) 72542 .IP \[bu] 2 72543 operations: 72544 .RS 2 72545 .IP \[bu] 2 72546 Run hashing operations in parallel (Nick Craig-Wood) 72547 .IP \[bu] 2 72548 Don\[aq]t calculate checksums when using \f[C]--ignore-checksum\f[R] 72549 (Nick Craig-Wood) 72550 .IP \[bu] 2 72551 Check transfer hashes when using \f[C]--size-only\f[R] mode (Nick 72552 Craig-Wood) 72553 .IP \[bu] 2 72554 Disable multi thread copy for local to local copies (Nick Craig-Wood) 72555 .IP \[bu] 2 72556 Debug successful hashes as well as failures (Nick Craig-Wood) 72557 .RE 72558 .IP \[bu] 2 72559 rc 72560 .RS 2 72561 .IP \[bu] 2 72562 Add ability to stop async jobs (Aleksandar Jankovic) 72563 .IP \[bu] 2 72564 Return current settings if core/bwlimit called without parameters (Nick 72565 Craig-Wood) 72566 .IP \[bu] 2 72567 Rclone-WebUI integration with rclone (Chaitanya Bankanhal) 72568 .IP \[bu] 2 72569 Added command line parameter to control the cross origin resource 72570 sharing (CORS) in the rcd. 72571 (Security Improvement) (Chaitanya Bankanhal) 72572 .IP \[bu] 2 72573 Add anchor tags to the docs so links are consistent (Nick Craig-Wood) 72574 .IP \[bu] 2 72575 Remove _async key from input parameters after parsing so later 72576 operations won\[aq]t get confused (buengese) 72577 .IP \[bu] 2 72578 Add call to clear stats (Aleksandar Jankovic) 72579 .RE 72580 .IP \[bu] 2 72581 rcd 72582 .RS 2 72583 .IP \[bu] 2 72584 Auto-login for web-gui (Chaitanya Bankanhal) 72585 .IP \[bu] 2 72586 Implement \f[C]--baseurl\f[R] for rcd and web-gui (Chaitanya Bankanhal) 72587 .RE 72588 .IP \[bu] 2 72589 serve dlna 72590 .RS 2 72591 .IP \[bu] 2 72592 Only select interfaces which can multicast for SSDP (Nick Craig-Wood) 72593 .IP \[bu] 2 72594 Add more builtin mime types to cover standard audio/video (Nick 72595 Craig-Wood) 72596 .IP \[bu] 2 72597 Fix missing mime types on Android causing missing videos (Nick 72598 Craig-Wood) 72599 .RE 72600 .IP \[bu] 2 72601 serve ftp 72602 .RS 2 72603 .IP \[bu] 2 72604 Refactor to bring into line with other serve commands (Nick Craig-Wood) 72605 .IP \[bu] 2 72606 Implement \f[C]--auth-proxy\f[R] (Nick Craig-Wood) 72607 .RE 72608 .IP \[bu] 2 72609 serve http: Implement \f[C]--baseurl\f[R] (Nick Craig-Wood) 72610 .IP \[bu] 2 72611 serve restic: Implement \f[C]--baseurl\f[R] (Nick Craig-Wood) 72612 .IP \[bu] 2 72613 serve sftp 72614 .RS 2 72615 .IP \[bu] 2 72616 Implement auth proxy (Nick Craig-Wood) 72617 .IP \[bu] 2 72618 Fix detection of whether server is authorized (Nick Craig-Wood) 72619 .RE 72620 .IP \[bu] 2 72621 serve webdav 72622 .RS 2 72623 .IP \[bu] 2 72624 Implement \f[C]--baseurl\f[R] (Nick Craig-Wood) 72625 .IP \[bu] 2 72626 Support \f[C]--auth-proxy\f[R] (Nick Craig-Wood) 72627 .RE 72628 .RE 72629 .IP \[bu] 2 72630 Bug Fixes 72631 .RS 2 72632 .IP \[bu] 2 72633 Make \[dq]bad record MAC\[dq] a retriable error (Nick Craig-Wood) 72634 .IP \[bu] 2 72635 copyurl: Fix copying files that return HTTP errors (Nick Craig-Wood) 72636 .IP \[bu] 2 72637 march: Fix checking sub-directories when using \f[C]--no-traverse\f[R] 72638 (buengese) 72639 .IP \[bu] 2 72640 rc 72641 .RS 2 72642 .IP \[bu] 2 72643 Fix unmarshalable http.AuthFn in options and put in test for 72644 marshalability (Nick Craig-Wood) 72645 .IP \[bu] 2 72646 Move job expire flags to rc to fix initialization problem (Nick 72647 Craig-Wood) 72648 .IP \[bu] 2 72649 Fix \f[C]--loopback\f[R] with rc/list and others (Nick Craig-Wood) 72650 .RE 72651 .IP \[bu] 2 72652 rcat: Fix slowdown on systems with multiple hashes (Nick Craig-Wood) 72653 .IP \[bu] 2 72654 rcd: Fix permissions problems on cache directory with web gui download 72655 (Nick Craig-Wood) 72656 .RE 72657 .IP \[bu] 2 72658 Mount 72659 .RS 2 72660 .IP \[bu] 2 72661 Default \f[C]--daemon-timeout\f[R] to 15 minutes on macOS and FreeBSD 72662 (Nick Craig-Wood) 72663 .IP \[bu] 2 72664 Update docs to show mounting from root OK for bucket-based (Nick 72665 Craig-Wood) 72666 .IP \[bu] 2 72667 Remove nonseekable flag from write files (Nick Craig-Wood) 72668 .RE 72669 .IP \[bu] 2 72670 VFS 72671 .RS 2 72672 .IP \[bu] 2 72673 Make write without cache more efficient (Nick Craig-Wood) 72674 .IP \[bu] 2 72675 Fix \f[C]--vfs-cache-mode minimal\f[R] and \f[C]writes\f[R] ignoring 72676 cached files (Nick Craig-Wood) 72677 .RE 72678 .IP \[bu] 2 72679 Local 72680 .RS 2 72681 .IP \[bu] 2 72682 Add \f[C]--local-case-sensitive\f[R] and 72683 \f[C]--local-case-insensitive\f[R] (Nick Craig-Wood) 72684 .IP \[bu] 2 72685 Avoid polluting page cache when uploading local files to remote backends 72686 (Micha\[/l] Matczuk) 72687 .IP \[bu] 2 72688 Don\[aq]t calculate any hashes by default (Nick Craig-Wood) 72689 .IP \[bu] 2 72690 Fadvise run syscall on a dedicated go routine (Micha\[/l] Matczuk) 72691 .RE 72692 .IP \[bu] 2 72693 Azure Blob 72694 .RS 2 72695 .IP \[bu] 2 72696 Azure Storage Emulator support (Sandeep) 72697 .IP \[bu] 2 72698 Updated config help details to remove connection string references 72699 (Sandeep) 72700 .IP \[bu] 2 72701 Make all operations work from the root (Nick Craig-Wood) 72702 .RE 72703 .IP \[bu] 2 72704 B2 72705 .RS 2 72706 .IP \[bu] 2 72707 Implement link sharing (yparitcher) 72708 .IP \[bu] 2 72709 Enable server-side copy to copy between buckets (Nick Craig-Wood) 72710 .IP \[bu] 2 72711 Make all operations work from the root (Nick Craig-Wood) 72712 .RE 72713 .IP \[bu] 2 72714 Drive 72715 .RS 2 72716 .IP \[bu] 2 72717 Fix server-side copy of big files (Nick Craig-Wood) 72718 .IP \[bu] 2 72719 Update API for teamdrive use (Nick Craig-Wood) 72720 .IP \[bu] 2 72721 Add error for purge with \f[C]--drive-trashed-only\f[R] (ginvine) 72722 .RE 72723 .IP \[bu] 2 72724 Fichier 72725 .RS 2 72726 .IP \[bu] 2 72727 Make FolderID int and adjust related code (buengese) 72728 .RE 72729 .IP \[bu] 2 72730 Google Cloud Storage 72731 .RS 2 72732 .IP \[bu] 2 72733 Reduce oauth scope requested as suggested by Google (Nick Craig-Wood) 72734 .IP \[bu] 2 72735 Make all operations work from the root (Nick Craig-Wood) 72736 .RE 72737 .IP \[bu] 2 72738 HTTP 72739 .RS 2 72740 .IP \[bu] 2 72741 Add \f[C]--http-headers\f[R] flag for setting arbitrary headers (Nick 72742 Craig-Wood) 72743 .RE 72744 .IP \[bu] 2 72745 Jottacloud 72746 .RS 2 72747 .IP \[bu] 2 72748 Use new api for retrieving internal username (buengese) 72749 .IP \[bu] 2 72750 Refactor configuration and minor cleanup (buengese) 72751 .RE 72752 .IP \[bu] 2 72753 Koofr 72754 .RS 2 72755 .IP \[bu] 2 72756 Support setting modification times on Koofr backend. 72757 (jaKa) 72758 .RE 72759 .IP \[bu] 2 72760 Opendrive 72761 .RS 2 72762 .IP \[bu] 2 72763 Refactor to use existing lib/rest facilities for uploads (Nick 72764 Craig-Wood) 72765 .RE 72766 .IP \[bu] 2 72767 Qingstor 72768 .RS 2 72769 .IP \[bu] 2 72770 Upgrade to v3 SDK and fix listing loop (Nick Craig-Wood) 72771 .IP \[bu] 2 72772 Make all operations work from the root (Nick Craig-Wood) 72773 .RE 72774 .IP \[bu] 2 72775 S3 72776 .RS 2 72777 .IP \[bu] 2 72778 Add INTELLIGENT_TIERING storage class (Matti Niemenmaa) 72779 .IP \[bu] 2 72780 Make all operations work from the root (Nick Craig-Wood) 72781 .RE 72782 .IP \[bu] 2 72783 SFTP 72784 .RS 2 72785 .IP \[bu] 2 72786 Add missing interface check and fix About (Nick Craig-Wood) 72787 .IP \[bu] 2 72788 Completely ignore all modtime checks if SetModTime=false (Jon Fautley) 72789 .IP \[bu] 2 72790 Support md5/sha1 with rsync.net (Nick Craig-Wood) 72791 .IP \[bu] 2 72792 Save the md5/sha1 command in use to the config file for efficiency (Nick 72793 Craig-Wood) 72794 .IP \[bu] 2 72795 Opt-in support for diffie-hellman-group-exchange-sha256 72796 diffie-hellman-group-exchange-sha1 (Yi FU) 72797 .RE 72798 .IP \[bu] 2 72799 Swift 72800 .RS 2 72801 .IP \[bu] 2 72802 Use FixRangeOption to fix 0 length files via the VFS (Nick Craig-Wood) 72803 .IP \[bu] 2 72804 Fix upload when using no_chunk to return the correct size (Nick 72805 Craig-Wood) 72806 .IP \[bu] 2 72807 Make all operations work from the root (Nick Craig-Wood) 72808 .IP \[bu] 2 72809 Fix segments leak during failed large file uploads. 72810 (nguyenhuuluan434) 72811 .RE 72812 .IP \[bu] 2 72813 WebDAV 72814 .RS 2 72815 .IP \[bu] 2 72816 Add \f[C]--webdav-bearer-token-command\f[R] (Nick Craig-Wood) 72817 .IP \[bu] 2 72818 Refresh token when it expires with 72819 \f[C]--webdav-bearer-token-command\f[R] (Nick Craig-Wood) 72820 .IP \[bu] 2 72821 Add docs for using bearer_token_command with oidc-agent (Paul Millar) 72822 .RE 72823 .SS v1.48.0 - 2019-06-15 72824 .IP \[bu] 2 72825 New commands 72826 .RS 2 72827 .IP \[bu] 2 72828 serve sftp: Serve an rclone remote over SFTP (Nick Craig-Wood) 72829 .RE 72830 .IP \[bu] 2 72831 New Features 72832 .RS 2 72833 .IP \[bu] 2 72834 Multi threaded downloads to local storage (Nick Craig-Wood) 72835 .RS 2 72836 .IP \[bu] 2 72837 controlled with \f[C]--multi-thread-cutoff\f[R] and 72838 \f[C]--multi-thread-streams\f[R] 72839 .RE 72840 .IP \[bu] 2 72841 Use rclone.conf from rclone executable directory to enable portable use 72842 (albertony) 72843 .IP \[bu] 2 72844 Allow sync of a file and a directory with the same name (forgems) 72845 .RS 2 72846 .IP \[bu] 2 72847 this is common on bucket-based remotes, e.g. 72848 s3, gcs 72849 .RE 72850 .IP \[bu] 2 72851 Add \f[C]--ignore-case-sync\f[R] for forced case insensitivity 72852 (garry415) 72853 .IP \[bu] 2 72854 Implement \f[C]--stats-one-line-date\f[R] and 72855 \f[C]--stats-one-line-date-format\f[R] (Peter Berbec) 72856 .IP \[bu] 2 72857 Log an ERROR for all commands which exit with non-zero status (Nick 72858 Craig-Wood) 72859 .IP \[bu] 2 72860 Use go-homedir to read the home directory more reliably (Nick 72861 Craig-Wood) 72862 .IP \[bu] 2 72863 Enable creating encrypted config through external script invocation 72864 (Wojciech Smigielski) 72865 .IP \[bu] 2 72866 build: Drop support for go1.8 (Nick Craig-Wood) 72867 .IP \[bu] 2 72868 config: Make config create/update encrypt passwords where necessary 72869 (Nick Craig-Wood) 72870 .IP \[bu] 2 72871 copyurl: Honor \f[C]--no-check-certificate\f[R] (Stefan Breunig) 72872 .IP \[bu] 2 72873 install: Linux skip man pages if no mandb (didil) 72874 .IP \[bu] 2 72875 lsf: Support showing the Tier of the object (Nick Craig-Wood) 72876 .IP \[bu] 2 72877 lsjson 72878 .RS 2 72879 .IP \[bu] 2 72880 Added EncryptedPath to output (calisro) 72881 .IP \[bu] 2 72882 Support showing the Tier of the object (Nick Craig-Wood) 72883 .IP \[bu] 2 72884 Add IsBucket field for bucket-based remote listing of the root (Nick 72885 Craig-Wood) 72886 .RE 72887 .IP \[bu] 2 72888 rc 72889 .RS 2 72890 .IP \[bu] 2 72891 Add \f[C]--loopback\f[R] flag to run commands directly without a server 72892 (Nick Craig-Wood) 72893 .IP \[bu] 2 72894 Add operations/fsinfo: Return information about the remote (Nick 72895 Craig-Wood) 72896 .IP \[bu] 2 72897 Skip auth for OPTIONS request (Nick Craig-Wood) 72898 .IP \[bu] 2 72899 cmd/providers: Add DefaultStr, ValueStr and Type fields (Nick 72900 Craig-Wood) 72901 .IP \[bu] 2 72902 jobs: Make job expiry timeouts configurable (Aleksandar Jankovic) 72903 .RE 72904 .IP \[bu] 2 72905 serve dlna reworked and improved (Dan Walters) 72906 .IP \[bu] 2 72907 serve ftp: add \f[C]--ftp-public-ip\f[R] flag to specify public IP 72908 (calistri) 72909 .IP \[bu] 2 72910 serve restic: Add support for \f[C]--private-repos\f[R] in 72911 \f[C]serve restic\f[R] (Florian Apolloner) 72912 .IP \[bu] 2 72913 serve webdav: Combine serve webdav and serve http (Gary Kim) 72914 .IP \[bu] 2 72915 size: Ignore negative sizes when calculating total (Garry McNulty) 72916 .RE 72917 .IP \[bu] 2 72918 Bug Fixes 72919 .RS 2 72920 .IP \[bu] 2 72921 Make move and copy individual files obey \f[C]--backup-dir\f[R] (Nick 72922 Craig-Wood) 72923 .IP \[bu] 2 72924 If \f[C]--ignore-checksum\f[R] is in effect, don\[aq]t calculate 72925 checksum (Nick Craig-Wood) 72926 .IP \[bu] 2 72927 moveto: Fix case-insensitive same remote move (Gary Kim) 72928 .IP \[bu] 2 72929 rc: Fix serving bucket-based objects with \f[C]--rc-serve\f[R] (Nick 72930 Craig-Wood) 72931 .IP \[bu] 2 72932 serve webdav: Fix serveDir not being updated with changes from webdav 72933 (Gary Kim) 72934 .RE 72935 .IP \[bu] 2 72936 Mount 72937 .RS 2 72938 .IP \[bu] 2 72939 Fix poll interval documentation (Animosity022) 72940 .RE 72941 .IP \[bu] 2 72942 VFS 72943 .RS 2 72944 .IP \[bu] 2 72945 Make WriteAt for non cached files work with non-sequential writes (Nick 72946 Craig-Wood) 72947 .RE 72948 .IP \[bu] 2 72949 Local 72950 .RS 2 72951 .IP \[bu] 2 72952 Only calculate the required hashes for big speedup (Nick Craig-Wood) 72953 .IP \[bu] 2 72954 Log errors when listing instead of returning an error (Nick Craig-Wood) 72955 .IP \[bu] 2 72956 Fix preallocate warning on Linux with ZFS (Nick Craig-Wood) 72957 .RE 72958 .IP \[bu] 2 72959 Crypt 72960 .RS 2 72961 .IP \[bu] 2 72962 Make rclone dedupe work through crypt (Nick Craig-Wood) 72963 .IP \[bu] 2 72964 Fix wrapping of ChangeNotify to decrypt directories properly (Nick 72965 Craig-Wood) 72966 .IP \[bu] 2 72967 Support PublicLink (rclone link) of underlying backend (Nick Craig-Wood) 72968 .IP \[bu] 2 72969 Implement Optional methods SetTier, GetTier (Nick Craig-Wood) 72970 .RE 72971 .IP \[bu] 2 72972 B2 72973 .RS 2 72974 .IP \[bu] 2 72975 Implement server-side copy (Nick Craig-Wood) 72976 .IP \[bu] 2 72977 Implement SetModTime (Nick Craig-Wood) 72978 .RE 72979 .IP \[bu] 2 72980 Drive 72981 .RS 2 72982 .IP \[bu] 2 72983 Fix move and copy from TeamDrive to GDrive (Fionera) 72984 .IP \[bu] 2 72985 Add notes that cleanup works in the background on drive (Nick 72986 Craig-Wood) 72987 .IP \[bu] 2 72988 Add \f[C]--drive-server-side-across-configs\f[R] to default back to old 72989 server-side copy semantics by default (Nick Craig-Wood) 72990 .IP \[bu] 2 72991 Add \f[C]--drive-size-as-quota\f[R] to show storage quota usage for file 72992 size (Garry McNulty) 72993 .RE 72994 .IP \[bu] 2 72995 FTP 72996 .RS 2 72997 .IP \[bu] 2 72998 Add FTP List timeout (Jeff Quinn) 72999 .IP \[bu] 2 73000 Add FTP over TLS support (Gary Kim) 73001 .IP \[bu] 2 73002 Add \f[C]--ftp-no-check-certificate\f[R] option for FTPS (Gary Kim) 73003 .RE 73004 .IP \[bu] 2 73005 Google Cloud Storage 73006 .RS 2 73007 .IP \[bu] 2 73008 Fix upload errors when uploading pre 1970 files (Nick Craig-Wood) 73009 .RE 73010 .IP \[bu] 2 73011 Jottacloud 73012 .RS 2 73013 .IP \[bu] 2 73014 Add support for selecting device and mountpoint. 73015 (buengese) 73016 .RE 73017 .IP \[bu] 2 73018 Mega 73019 .RS 2 73020 .IP \[bu] 2 73021 Add cleanup support (Gary Kim) 73022 .RE 73023 .IP \[bu] 2 73024 Onedrive 73025 .RS 2 73026 .IP \[bu] 2 73027 More accurately check if root is found (Cnly) 73028 .RE 73029 .IP \[bu] 2 73030 S3 73031 .RS 2 73032 .IP \[bu] 2 73033 Support S3 Accelerated endpoints with 73034 \f[C]--s3-use-accelerate-endpoint\f[R] (Nick Craig-Wood) 73035 .IP \[bu] 2 73036 Add config info for Wasabi\[aq]s EU Central endpoint (Robert Marko) 73037 .IP \[bu] 2 73038 Make SetModTime work for GLACIER while syncing (Philip Harvey) 73039 .RE 73040 .IP \[bu] 2 73041 SFTP 73042 .RS 2 73043 .IP \[bu] 2 73044 Add About support (Gary Kim) 73045 .IP \[bu] 2 73046 Fix about parsing of \f[C]df\f[R] results so it can cope with -ve 73047 results (Nick Craig-Wood) 73048 .IP \[bu] 2 73049 Send custom client version and debug server version (Nick Craig-Wood) 73050 .RE 73051 .IP \[bu] 2 73052 WebDAV 73053 .RS 2 73054 .IP \[bu] 2 73055 Retry on 423 Locked errors (Nick Craig-Wood) 73056 .RE 73057 .SS v1.47.0 - 2019-04-13 73058 .IP \[bu] 2 73059 New backends 73060 .RS 2 73061 .IP \[bu] 2 73062 Backend for Koofr cloud storage service. 73063 (jaKa) 73064 .RE 73065 .IP \[bu] 2 73066 New Features 73067 .RS 2 73068 .IP \[bu] 2 73069 Resume downloads if the reader fails in copy (Nick Craig-Wood) 73070 .RS 2 73071 .IP \[bu] 2 73072 this means rclone will restart transfers if the source has an error 73073 .IP \[bu] 2 73074 this is most useful for downloads or cloud to cloud copies 73075 .RE 73076 .IP \[bu] 2 73077 Use \f[C]--fast-list\f[R] for listing operations where it won\[aq]t use 73078 more memory (Nick Craig-Wood) 73079 .RS 2 73080 .IP \[bu] 2 73081 this should speed up the following operations on remotes which support 73082 \f[C]ListR\f[R] 73083 .IP \[bu] 2 73084 \f[C]dedupe\f[R], \f[C]serve restic\f[R] \f[C]lsf\f[R], \f[C]ls\f[R], 73085 \f[C]lsl\f[R], \f[C]lsjson\f[R], \f[C]lsd\f[R], \f[C]md5sum\f[R], 73086 \f[C]sha1sum\f[R], \f[C]hashsum\f[R], \f[C]size\f[R], \f[C]delete\f[R], 73087 \f[C]cat\f[R], \f[C]settier\f[R] 73088 .IP \[bu] 2 73089 use \f[C]--disable ListR\f[R] to get old behaviour if required 73090 .RE 73091 .IP \[bu] 2 73092 Make \f[C]--files-from\f[R] traverse the destination unless 73093 \f[C]--no-traverse\f[R] is set (Nick Craig-Wood) 73094 .RS 2 73095 .IP \[bu] 2 73096 this fixes \f[C]--files-from\f[R] with Google drive and excessive API 73097 use in general. 73098 .RE 73099 .IP \[bu] 2 73100 Make server-side copy account bytes and obey \f[C]--max-transfer\f[R] 73101 (Nick Craig-Wood) 73102 .IP \[bu] 2 73103 Add \f[C]--create-empty-src-dirs\f[R] flag and default to not creating 73104 empty dirs (ishuah) 73105 .IP \[bu] 2 73106 Add client side TLS/SSL flags 73107 \f[C]--ca-cert\f[R]/\f[C]--client-cert\f[R]/\f[C]--client-key\f[R] (Nick 73108 Craig-Wood) 73109 .IP \[bu] 2 73110 Implement \f[C]--suffix-keep-extension\f[R] for use with 73111 \f[C]--suffix\f[R] (Nick Craig-Wood) 73112 .IP \[bu] 2 73113 build: 73114 .RS 2 73115 .IP \[bu] 2 73116 Switch to semver compliant version tags to be go modules compliant (Nick 73117 Craig-Wood) 73118 .IP \[bu] 2 73119 Update to use go1.12.x for the build (Nick Craig-Wood) 73120 .RE 73121 .IP \[bu] 2 73122 serve dlna: Add connection manager service description to improve 73123 compatibility (Dan Walters) 73124 .IP \[bu] 2 73125 lsf: Add \[aq]e\[aq] format to show encrypted names and \[aq]o\[aq] for 73126 original IDs (Nick Craig-Wood) 73127 .IP \[bu] 2 73128 lsjson: Added \f[C]--files-only\f[R] and \f[C]--dirs-only\f[R] flags 73129 (calistri) 73130 .IP \[bu] 2 73131 rc: Implement operations/publiclink the equivalent of 73132 \f[C]rclone link\f[R] (Nick Craig-Wood) 73133 .RE 73134 .IP \[bu] 2 73135 Bug Fixes 73136 .RS 2 73137 .IP \[bu] 2 73138 accounting: Fix total ETA when \f[C]--stats-unit bits\f[R] is in effect 73139 (Nick Craig-Wood) 73140 .IP \[bu] 2 73141 Bash TAB completion 73142 .RS 2 73143 .IP \[bu] 2 73144 Use private custom func to fix clash between rclone and kubectl (Nick 73145 Craig-Wood) 73146 .IP \[bu] 2 73147 Fix for remotes with underscores in their names (Six) 73148 .IP \[bu] 2 73149 Fix completion of remotes (Florian Gamb\[:o]ck) 73150 .IP \[bu] 2 73151 Fix autocompletion of remote paths with spaces (Danil Semelenov) 73152 .RE 73153 .IP \[bu] 2 73154 serve dlna: Fix root XML service descriptor (Dan Walters) 73155 .IP \[bu] 2 73156 ncdu: Fix display corruption with Chinese characters (Nick Craig-Wood) 73157 .IP \[bu] 2 73158 Add SIGTERM to signals which run the exit handlers on unix (Nick 73159 Craig-Wood) 73160 .IP \[bu] 2 73161 rc: Reload filter when the options are set via the rc (Nick Craig-Wood) 73162 .RE 73163 .IP \[bu] 2 73164 VFS / Mount 73165 .RS 2 73166 .IP \[bu] 2 73167 Fix FreeBSD: Ignore Truncate if called with no readers and already the 73168 correct size (Nick Craig-Wood) 73169 .IP \[bu] 2 73170 Read directory and check for a file before mkdir (Nick Craig-Wood) 73171 .IP \[bu] 2 73172 Shorten the locking window for vfs/refresh (Nick Craig-Wood) 73173 .RE 73174 .IP \[bu] 2 73175 Azure Blob 73176 .RS 2 73177 .IP \[bu] 2 73178 Enable MD5 checksums when uploading files bigger than the 73179 \[dq]Cutoff\[dq] (Dr.Rx) 73180 .IP \[bu] 2 73181 Fix SAS URL support (Nick Craig-Wood) 73182 .RE 73183 .IP \[bu] 2 73184 B2 73185 .RS 2 73186 .IP \[bu] 2 73187 Allow manual configuration of backblaze downloadUrl (Vince) 73188 .IP \[bu] 2 73189 Ignore already_hidden error on remove (Nick Craig-Wood) 73190 .IP \[bu] 2 73191 Ignore malformed \f[C]src_last_modified_millis\f[R] (Nick Craig-Wood) 73192 .RE 73193 .IP \[bu] 2 73194 Drive 73195 .RS 2 73196 .IP \[bu] 2 73197 Add \f[C]--skip-checksum-gphotos\f[R] to ignore incorrect checksums on 73198 Google Photos (Nick Craig-Wood) 73199 .IP \[bu] 2 73200 Allow server-side move/copy between different remotes. 73201 (Fionera) 73202 .IP \[bu] 2 73203 Add docs on team drives and \f[C]--fast-list\f[R] eventual consistency 73204 (Nestar47) 73205 .IP \[bu] 2 73206 Fix imports of text files (Nick Craig-Wood) 73207 .IP \[bu] 2 73208 Fix range requests on 0 length files (Nick Craig-Wood) 73209 .IP \[bu] 2 73210 Fix creation of duplicates with server-side copy (Nick Craig-Wood) 73211 .RE 73212 .IP \[bu] 2 73213 Dropbox 73214 .RS 2 73215 .IP \[bu] 2 73216 Retry blank errors to fix long listings (Nick Craig-Wood) 73217 .RE 73218 .IP \[bu] 2 73219 FTP 73220 .RS 2 73221 .IP \[bu] 2 73222 Add \f[C]--ftp-concurrency\f[R] to limit maximum number of connections 73223 (Nick Craig-Wood) 73224 .RE 73225 .IP \[bu] 2 73226 Google Cloud Storage 73227 .RS 2 73228 .IP \[bu] 2 73229 Fall back to default application credentials (marcintustin) 73230 .IP \[bu] 2 73231 Allow bucket policy only buckets (Nick Craig-Wood) 73232 .RE 73233 .IP \[bu] 2 73234 HTTP 73235 .RS 2 73236 .IP \[bu] 2 73237 Add \f[C]--http-no-slash\f[R] for websites with directories with no 73238 slashes (Nick Craig-Wood) 73239 .IP \[bu] 2 73240 Remove duplicates from listings (Nick Craig-Wood) 73241 .IP \[bu] 2 73242 Fix socket leak on 404 errors (Nick Craig-Wood) 73243 .RE 73244 .IP \[bu] 2 73245 Jottacloud 73246 .RS 2 73247 .IP \[bu] 2 73248 Fix token refresh (Sebastian B\[:u]nger) 73249 .IP \[bu] 2 73250 Add device registration (Oliver Heyme) 73251 .RE 73252 .IP \[bu] 2 73253 Onedrive 73254 .RS 2 73255 .IP \[bu] 2 73256 Implement graceful cancel of multipart uploads if rclone is interrupted 73257 (Cnly) 73258 .IP \[bu] 2 73259 Always add trailing colon to path when addressing items, (Cnly) 73260 .IP \[bu] 2 73261 Return errors instead of panic for invalid uploads (Fabian M\[:o]ller) 73262 .RE 73263 .IP \[bu] 2 73264 S3 73265 .RS 2 73266 .IP \[bu] 2 73267 Add support for \[dq]Glacier Deep Archive\[dq] storage class (Manu) 73268 .IP \[bu] 2 73269 Update Dreamhost endpoint (Nick Craig-Wood) 73270 .IP \[bu] 2 73271 Note incompatibility with CEPH Jewel (Nick Craig-Wood) 73272 .RE 73273 .IP \[bu] 2 73274 SFTP 73275 .RS 2 73276 .IP \[bu] 2 73277 Allow custom ssh client config (Alexandru Bumbacea) 73278 .RE 73279 .IP \[bu] 2 73280 Swift 73281 .RS 2 73282 .IP \[bu] 2 73283 Obey Retry-After to enable OVH restore from cold storage (Nick 73284 Craig-Wood) 73285 .IP \[bu] 2 73286 Work around token expiry on CEPH (Nick Craig-Wood) 73287 .RE 73288 .IP \[bu] 2 73289 WebDAV 73290 .RS 2 73291 .IP \[bu] 2 73292 Allow IsCollection property to be integer or boolean (Nick Craig-Wood) 73293 .IP \[bu] 2 73294 Fix race when creating directories (Nick Craig-Wood) 73295 .IP \[bu] 2 73296 Fix About/df when reading the available/total returns 0 (Nick 73297 Craig-Wood) 73298 .RE 73299 .SS v1.46 - 2019-02-09 73300 .IP \[bu] 2 73301 New backends 73302 .RS 2 73303 .IP \[bu] 2 73304 Support Alibaba Cloud (Aliyun) OSS via the s3 backend (Nick Craig-Wood) 73305 .RE 73306 .IP \[bu] 2 73307 New commands 73308 .RS 2 73309 .IP \[bu] 2 73310 serve dlna: serves a remove via DLNA for the local network (nicolov) 73311 .RE 73312 .IP \[bu] 2 73313 New Features 73314 .RS 2 73315 .IP \[bu] 2 73316 copy, move: Restore deprecated \f[C]--no-traverse\f[R] flag (Nick 73317 Craig-Wood) 73318 .RS 2 73319 .IP \[bu] 2 73320 This is useful for when transferring a small number of files into a 73321 large destination 73322 .RE 73323 .IP \[bu] 2 73324 genautocomplete: Add remote path completion for bash completion 73325 (Christopher Peterson & Danil Semelenov) 73326 .IP \[bu] 2 73327 Buffer memory handling reworked to return memory to the OS better (Nick 73328 Craig-Wood) 73329 .RS 2 73330 .IP \[bu] 2 73331 Buffer recycling library to replace sync.Pool 73332 .IP \[bu] 2 73333 Optionally use memory mapped memory for better memory shrinking 73334 .IP \[bu] 2 73335 Enable with \f[C]--use-mmap\f[R] if having memory problems - not default 73336 yet 73337 .RE 73338 .IP \[bu] 2 73339 Parallelise reading of files specified by \f[C]--files-from\f[R] (Nick 73340 Craig-Wood) 73341 .IP \[bu] 2 73342 check: Add stats showing total files matched. 73343 (Dario Guzik) 73344 .IP \[bu] 2 73345 Allow rename/delete open files under Windows (Nick Craig-Wood) 73346 .IP \[bu] 2 73347 lsjson: Use exactly the correct number of decimal places in the seconds 73348 (Nick Craig-Wood) 73349 .IP \[bu] 2 73350 Add cookie support with cmdline switch \f[C]--use-cookies\f[R] for all 73351 HTTP based remotes (qip) 73352 .IP \[bu] 2 73353 Warn if \f[C]--checksum\f[R] is set but there are no hashes available 73354 (Nick Craig-Wood) 73355 .IP \[bu] 2 73356 Rework rate limiting (pacer) to be more accurate and allow bursting 73357 (Nick Craig-Wood) 73358 .IP \[bu] 2 73359 Improve error reporting for too many/few arguments in commands (Nick 73360 Craig-Wood) 73361 .IP \[bu] 2 73362 listremotes: Remove \f[C]-l\f[R] short flag as it conflicts with the new 73363 global flag (weetmuts) 73364 .IP \[bu] 2 73365 Make http serving with auth generate INFO messages on auth fail (Nick 73366 Craig-Wood) 73367 .RE 73368 .IP \[bu] 2 73369 Bug Fixes 73370 .RS 2 73371 .IP \[bu] 2 73372 Fix layout of stats (Nick Craig-Wood) 73373 .IP \[bu] 2 73374 Fix \f[C]--progress\f[R] crash under Windows Jenkins (Nick Craig-Wood) 73375 .IP \[bu] 2 73376 Fix transfer of google/onedrive docs by calling Rcat in Copy when size 73377 is -1 (Cnly) 73378 .IP \[bu] 2 73379 copyurl: Fix checking of \f[C]--dry-run\f[R] (Denis Skovpen) 73380 .RE 73381 .IP \[bu] 2 73382 Mount 73383 .RS 2 73384 .IP \[bu] 2 73385 Check that mountpoint and local directory to mount don\[aq]t overlap 73386 (Nick Craig-Wood) 73387 .IP \[bu] 2 73388 Fix mount size under 32 bit Windows (Nick Craig-Wood) 73389 .RE 73390 .IP \[bu] 2 73391 VFS 73392 .RS 2 73393 .IP \[bu] 2 73394 Implement renaming of directories for backends without DirMove (Nick 73395 Craig-Wood) 73396 .RS 2 73397 .IP \[bu] 2 73398 now all backends except b2 support renaming directories 73399 .RE 73400 .IP \[bu] 2 73401 Implement \f[C]--vfs-cache-max-size\f[R] to limit the total size of the 73402 cache (Nick Craig-Wood) 73403 .IP \[bu] 2 73404 Add \f[C]--dir-perms\f[R] and \f[C]--file-perms\f[R] flags to set 73405 default permissions (Nick Craig-Wood) 73406 .IP \[bu] 2 73407 Fix deadlock on concurrent operations on a directory (Nick Craig-Wood) 73408 .IP \[bu] 2 73409 Fix deadlock between RWFileHandle.close and File.Remove (Nick 73410 Craig-Wood) 73411 .IP \[bu] 2 73412 Fix renaming/deleting open files with cache mode \[dq]writes\[dq] under 73413 Windows (Nick Craig-Wood) 73414 .IP \[bu] 2 73415 Fix panic on rename with \f[C]--dry-run\f[R] set (Nick Craig-Wood) 73416 .IP \[bu] 2 73417 Fix vfs/refresh with recurse=true needing the \f[C]--fast-list\f[R] flag 73418 .RE 73419 .IP \[bu] 2 73420 Local 73421 .RS 2 73422 .IP \[bu] 2 73423 Add support for \f[C]-l\f[R]/\f[C]--links\f[R] (symbolic link 73424 translation) (yair\[at]unicorn) 73425 .RS 2 73426 .IP \[bu] 2 73427 this works by showing links as \f[C]link.rclonelink\f[R] - see local 73428 backend docs for more info 73429 .IP \[bu] 2 73430 this errors if used with \f[C]-L\f[R]/\f[C]--copy-links\f[R] 73431 .RE 73432 .IP \[bu] 2 73433 Fix renaming/deleting open files on Windows (Nick Craig-Wood) 73434 .RE 73435 .IP \[bu] 2 73436 Crypt 73437 .RS 2 73438 .IP \[bu] 2 73439 Check for maximum length before decrypting filename to fix panic (Garry 73440 McNulty) 73441 .RE 73442 .IP \[bu] 2 73443 Azure Blob 73444 .RS 2 73445 .IP \[bu] 2 73446 Allow building azureblob backend on *BSD (themylogin) 73447 .IP \[bu] 2 73448 Use the rclone HTTP client to support \f[C]--dump headers\f[R], 73449 \f[C]--tpslimit\f[R], etc. 73450 (Nick Craig-Wood) 73451 .IP \[bu] 2 73452 Use the s3 pacer for 0 delay in non error conditions (Nick Craig-Wood) 73453 .IP \[bu] 2 73454 Ignore directory markers (Nick Craig-Wood) 73455 .IP \[bu] 2 73456 Stop Mkdir attempting to create existing containers (Nick Craig-Wood) 73457 .RE 73458 .IP \[bu] 2 73459 B2 73460 .RS 2 73461 .IP \[bu] 2 73462 cleanup: will remove unfinished large files >24hrs old (Garry McNulty) 73463 .IP \[bu] 2 73464 For a bucket limited application key check the bucket name (Nick 73465 Craig-Wood) 73466 .RS 2 73467 .IP \[bu] 2 73468 before this, rclone would use the authorised bucket regardless of what 73469 you put on the command line 73470 .RE 73471 .IP \[bu] 2 73472 Added \f[C]--b2-disable-checksum\f[R] flag (Wojciech Smigielski) 73473 .RS 2 73474 .IP \[bu] 2 73475 this enables large files to be uploaded without a SHA-1 hash for speed 73476 reasons 73477 .RE 73478 .RE 73479 .IP \[bu] 2 73480 Drive 73481 .RS 2 73482 .IP \[bu] 2 73483 Set default pacer to 100ms for 10 tps (Nick Craig-Wood) 73484 .RS 2 73485 .IP \[bu] 2 73486 This fits the Google defaults much better and reduces the 403 errors 73487 massively 73488 .IP \[bu] 2 73489 Add \f[C]--drive-pacer-min-sleep\f[R] and \f[C]--drive-pacer-burst\f[R] 73490 to control the pacer 73491 .RE 73492 .IP \[bu] 2 73493 Improve ChangeNotify support for items with multiple parents (Fabian 73494 M\[:o]ller) 73495 .IP \[bu] 2 73496 Fix ListR for items with multiple parents - this fixes oddities with 73497 \f[C]vfs/refresh\f[R] (Fabian M\[:o]ller) 73498 .IP \[bu] 2 73499 Fix using \f[C]--drive-impersonate\f[R] and appfolders (Nick Craig-Wood) 73500 .IP \[bu] 2 73501 Fix google docs in rclone mount for some (not all) applications (Nick 73502 Craig-Wood) 73503 .RE 73504 .IP \[bu] 2 73505 Dropbox 73506 .RS 2 73507 .IP \[bu] 2 73508 Retry-After support for Dropbox backend (Mathieu Carbou) 73509 .RE 73510 .IP \[bu] 2 73511 FTP 73512 .RS 2 73513 .IP \[bu] 2 73514 Wait for 60 seconds for a connection to Close then declare it dead (Nick 73515 Craig-Wood) 73516 .RS 2 73517 .IP \[bu] 2 73518 helps with indefinite hangs on some FTP servers 73519 .RE 73520 .RE 73521 .IP \[bu] 2 73522 Google Cloud Storage 73523 .RS 2 73524 .IP \[bu] 2 73525 Update google cloud storage endpoints (weetmuts) 73526 .RE 73527 .IP \[bu] 2 73528 HTTP 73529 .RS 2 73530 .IP \[bu] 2 73531 Add an example with username and password which is supported but 73532 wasn\[aq]t documented (Nick Craig-Wood) 73533 .IP \[bu] 2 73534 Fix backend with \f[C]--files-from\f[R] and nonexistent files (Nick 73535 Craig-Wood) 73536 .RE 73537 .IP \[bu] 2 73538 Hubic 73539 .RS 2 73540 .IP \[bu] 2 73541 Make error message more informative if authentication fails (Nick 73542 Craig-Wood) 73543 .RE 73544 .IP \[bu] 2 73545 Jottacloud 73546 .RS 2 73547 .IP \[bu] 2 73548 Resume and deduplication support (Oliver Heyme) 73549 .IP \[bu] 2 73550 Use token auth for all API requests Don\[aq]t store password anymore 73551 (Sebastian B\[:u]nger) 73552 .IP \[bu] 2 73553 Add support for 2-factor authentication (Sebastian B\[:u]nger) 73554 .RE 73555 .IP \[bu] 2 73556 Mega 73557 .RS 2 73558 .IP \[bu] 2 73559 Implement v2 account login which fixes logins for newer Mega accounts 73560 (Nick Craig-Wood) 73561 .IP \[bu] 2 73562 Return error if an unknown length file is attempted to be uploaded (Nick 73563 Craig-Wood) 73564 .IP \[bu] 2 73565 Add new error codes for better error reporting (Nick Craig-Wood) 73566 .RE 73567 .IP \[bu] 2 73568 Onedrive 73569 .RS 2 73570 .IP \[bu] 2 73571 Fix broken support for \[dq]shared with me\[dq] folders (Alex Chen) 73572 .IP \[bu] 2 73573 Fix root ID not normalised (Cnly) 73574 .IP \[bu] 2 73575 Return err instead of panic on unknown-sized uploads (Cnly) 73576 .RE 73577 .IP \[bu] 2 73578 Qingstor 73579 .RS 2 73580 .IP \[bu] 2 73581 Fix go routine leak on multipart upload errors (Nick Craig-Wood) 73582 .IP \[bu] 2 73583 Add upload chunk size/concurrency/cutoff control (Nick Craig-Wood) 73584 .IP \[bu] 2 73585 Default \f[C]--qingstor-upload-concurrency\f[R] to 1 to work around bug 73586 (Nick Craig-Wood) 73587 .RE 73588 .IP \[bu] 2 73589 S3 73590 .RS 2 73591 .IP \[bu] 2 73592 Implement \f[C]--s3-upload-cutoff\f[R] for single part uploads below 73593 this (Nick Craig-Wood) 73594 .IP \[bu] 2 73595 Change \f[C]--s3-upload-concurrency\f[R] default to 4 to increase 73596 performance (Nick Craig-Wood) 73597 .IP \[bu] 2 73598 Add \f[C]--s3-bucket-acl\f[R] to control bucket ACL (Nick Craig-Wood) 73599 .IP \[bu] 2 73600 Auto detect region for buckets on operation failure (Nick Craig-Wood) 73601 .IP \[bu] 2 73602 Add GLACIER storage class (William Cocker) 73603 .IP \[bu] 2 73604 Add Scaleway to s3 documentation (R\['e]my L\['e]one) 73605 .IP \[bu] 2 73606 Add AWS endpoint eu-north-1 (weetmuts) 73607 .RE 73608 .IP \[bu] 2 73609 SFTP 73610 .RS 2 73611 .IP \[bu] 2 73612 Add support for PEM encrypted private keys (Fabian M\[:o]ller) 73613 .IP \[bu] 2 73614 Add option to force the usage of an ssh-agent (Fabian M\[:o]ller) 73615 .IP \[bu] 2 73616 Perform environment variable expansion on key-file (Fabian M\[:o]ller) 73617 .IP \[bu] 2 73618 Fix rmdir on Windows based servers (e.g. 73619 CrushFTP) (Nick Craig-Wood) 73620 .IP \[bu] 2 73621 Fix rmdir deleting directory contents on some SFTP servers (Nick 73622 Craig-Wood) 73623 .IP \[bu] 2 73624 Fix error on dangling symlinks (Nick Craig-Wood) 73625 .RE 73626 .IP \[bu] 2 73627 Swift 73628 .RS 2 73629 .IP \[bu] 2 73630 Add \f[C]--swift-no-chunk\f[R] to disable segmented uploads in 73631 rcat/mount (Nick Craig-Wood) 73632 .IP \[bu] 2 73633 Introduce application credential auth support (kayrus) 73634 .IP \[bu] 2 73635 Fix memory usage by slimming Object (Nick Craig-Wood) 73636 .IP \[bu] 2 73637 Fix extra requests on upload (Nick Craig-Wood) 73638 .IP \[bu] 2 73639 Fix reauth on big files (Nick Craig-Wood) 73640 .RE 73641 .IP \[bu] 2 73642 Union 73643 .RS 2 73644 .IP \[bu] 2 73645 Fix poll-interval not working (Nick Craig-Wood) 73646 .RE 73647 .IP \[bu] 2 73648 WebDAV 73649 .RS 2 73650 .IP \[bu] 2 73651 Support About which means rclone mount will show the correct disk size 73652 (Nick Craig-Wood) 73653 .IP \[bu] 2 73654 Support MD5 and SHA1 hashes with Owncloud and Nextcloud (Nick 73655 Craig-Wood) 73656 .IP \[bu] 2 73657 Fail soft on time parsing errors (Nick Craig-Wood) 73658 .IP \[bu] 2 73659 Fix infinite loop on failed directory creation (Nick Craig-Wood) 73660 .IP \[bu] 2 73661 Fix identification of directories for Bitrix Site Manager (Nick 73662 Craig-Wood) 73663 .IP \[bu] 2 73664 Fix upload of 0 length files on some servers (Nick Craig-Wood) 73665 .IP \[bu] 2 73666 Fix if MKCOL fails with 423 Locked assume the directory exists (Nick 73667 Craig-Wood) 73668 .RE 73669 .SS v1.45 - 2018-11-24 73670 .IP \[bu] 2 73671 New backends 73672 .RS 2 73673 .IP \[bu] 2 73674 The Yandex backend was re-written - see below for details (Sebastian 73675 B\[:u]nger) 73676 .RE 73677 .IP \[bu] 2 73678 New commands 73679 .RS 2 73680 .IP \[bu] 2 73681 rcd: New command just to serve the remote control API (Nick Craig-Wood) 73682 .RE 73683 .IP \[bu] 2 73684 New Features 73685 .RS 2 73686 .IP \[bu] 2 73687 The remote control API (rc) was greatly expanded to allow full control 73688 over rclone (Nick Craig-Wood) 73689 .RS 2 73690 .IP \[bu] 2 73691 sensitive operations require authorization or the \f[C]--rc-no-auth\f[R] 73692 flag 73693 .IP \[bu] 2 73694 config/* operations to configure rclone 73695 .IP \[bu] 2 73696 options/* for reading/setting command line flags 73697 .IP \[bu] 2 73698 operations/* for all low level operations, e.g. 73699 copy file, list directory 73700 .IP \[bu] 2 73701 sync/* for sync, copy and move 73702 .IP \[bu] 2 73703 \f[C]--rc-files\f[R] flag to serve files on the rc http server 73704 .RS 2 73705 .IP \[bu] 2 73706 this is for building web native GUIs for rclone 73707 .RE 73708 .IP \[bu] 2 73709 Optionally serving objects on the rc http server 73710 .IP \[bu] 2 73711 Ensure rclone fails to start up if the \f[C]--rc\f[R] port is in use 73712 already 73713 .IP \[bu] 2 73714 See the rc docs (https://rclone.org/rc/) for more info 73715 .RE 73716 .IP \[bu] 2 73717 sync/copy/move 73718 .RS 2 73719 .IP \[bu] 2 73720 Make \f[C]--files-from\f[R] only read the objects specified and 73721 don\[aq]t scan directories (Nick Craig-Wood) 73722 .RS 2 73723 .IP \[bu] 2 73724 This is a huge speed improvement for destinations with lots of files 73725 .RE 73726 .RE 73727 .IP \[bu] 2 73728 filter: Add \f[C]--ignore-case\f[R] flag (Nick Craig-Wood) 73729 .IP \[bu] 2 73730 ncdu: Add remove function (\[aq]d\[aq] key) (Henning Surmeier) 73731 .IP \[bu] 2 73732 rc command 73733 .RS 2 73734 .IP \[bu] 2 73735 Add \f[C]--json\f[R] flag for structured JSON input (Nick Craig-Wood) 73736 .IP \[bu] 2 73737 Add \f[C]--user\f[R] and \f[C]--pass\f[R] flags and interpret 73738 \f[C]--rc-user\f[R], \f[C]--rc-pass\f[R], \f[C]--rc-addr\f[R] (Nick 73739 Craig-Wood) 73740 .RE 73741 .IP \[bu] 2 73742 build 73743 .RS 2 73744 .IP \[bu] 2 73745 Require go1.8 or later for compilation (Nick Craig-Wood) 73746 .IP \[bu] 2 73747 Enable softfloat on MIPS arch (Scott Edlund) 73748 .IP \[bu] 2 73749 Integration test framework revamped with a better report and better 73750 retries (Nick Craig-Wood) 73751 .RE 73752 .RE 73753 .IP \[bu] 2 73754 Bug Fixes 73755 .RS 2 73756 .IP \[bu] 2 73757 cmd: Make \f[C]--progress\f[R] update the stats correctly at the end 73758 (Nick Craig-Wood) 73759 .IP \[bu] 2 73760 config: Create config directory on save if it is missing (Nick 73761 Craig-Wood) 73762 .IP \[bu] 2 73763 dedupe: Check for existing filename before renaming a dupe file (ssaqua) 73764 .IP \[bu] 2 73765 move: Don\[aq]t create directories with \f[C]--dry-run\f[R] (Nick 73766 Craig-Wood) 73767 .IP \[bu] 2 73768 operations: Fix Purge and Rmdirs when dir is not the root (Nick 73769 Craig-Wood) 73770 .IP \[bu] 2 73771 serve http/webdav/restic: Ensure rclone exits if the port is in use 73772 (Nick Craig-Wood) 73773 .RE 73774 .IP \[bu] 2 73775 Mount 73776 .RS 2 73777 .IP \[bu] 2 73778 Make \f[C]--volname\f[R] work for Windows and macOS (Nick Craig-Wood) 73779 .RE 73780 .IP \[bu] 2 73781 Azure Blob 73782 .RS 2 73783 .IP \[bu] 2 73784 Avoid context deadline exceeded error by setting a large TryTimeout 73785 value (brused27) 73786 .IP \[bu] 2 73787 Fix erroneous Rmdir error \[dq]directory not empty\[dq] (Nick 73788 Craig-Wood) 73789 .IP \[bu] 2 73790 Wait for up to 60s to create a just deleted container (Nick Craig-Wood) 73791 .RE 73792 .IP \[bu] 2 73793 Dropbox 73794 .RS 2 73795 .IP \[bu] 2 73796 Add dropbox impersonate support (Jake Coggiano) 73797 .RE 73798 .IP \[bu] 2 73799 Jottacloud 73800 .RS 2 73801 .IP \[bu] 2 73802 Fix bug in \f[C]--fast-list\f[R] handing of empty folders (albertony) 73803 .RE 73804 .IP \[bu] 2 73805 Opendrive 73806 .RS 2 73807 .IP \[bu] 2 73808 Fix transfer of files with \f[C]+\f[R] and \f[C]&\f[R] in (Nick 73809 Craig-Wood) 73810 .IP \[bu] 2 73811 Fix retries of upload chunks (Nick Craig-Wood) 73812 .RE 73813 .IP \[bu] 2 73814 S3 73815 .RS 2 73816 .IP \[bu] 2 73817 Set ACL for server-side copies to that provided by the user (Nick 73818 Craig-Wood) 73819 .IP \[bu] 2 73820 Fix role_arn, credential_source, ... 73821 (Erik Swanson) 73822 .IP \[bu] 2 73823 Add config info for Wasabi\[aq]s US-West endpoint (Henry Ptasinski) 73824 .RE 73825 .IP \[bu] 2 73826 SFTP 73827 .RS 2 73828 .IP \[bu] 2 73829 Ensure file hash checking is really disabled (Jon Fautley) 73830 .RE 73831 .IP \[bu] 2 73832 Swift 73833 .RS 2 73834 .IP \[bu] 2 73835 Add pacer for retries to make swift more reliable (Nick Craig-Wood) 73836 .RE 73837 .IP \[bu] 2 73838 WebDAV 73839 .RS 2 73840 .IP \[bu] 2 73841 Add Content-Type to PUT requests (Nick Craig-Wood) 73842 .IP \[bu] 2 73843 Fix config parsing so \f[C]--webdav-user\f[R] and 73844 \f[C]--webdav-pass\f[R] flags work (Nick Craig-Wood) 73845 .IP \[bu] 2 73846 Add RFC3339 date format (Ralf Hemberger) 73847 .RE 73848 .IP \[bu] 2 73849 Yandex 73850 .RS 2 73851 .IP \[bu] 2 73852 The yandex backend was re-written (Sebastian B\[:u]nger) 73853 .RS 2 73854 .IP \[bu] 2 73855 This implements low level retries (Sebastian B\[:u]nger) 73856 .IP \[bu] 2 73857 Copy, Move, DirMove, PublicLink and About optional interfaces (Sebastian 73858 B\[:u]nger) 73859 .IP \[bu] 2 73860 Improved general error handling (Sebastian B\[:u]nger) 73861 .IP \[bu] 2 73862 Removed ListR for now due to inconsistent behaviour (Sebastian 73863 B\[:u]nger) 73864 .RE 73865 .RE 73866 .SS v1.44 - 2018-10-15 73867 .IP \[bu] 2 73868 New commands 73869 .RS 2 73870 .IP \[bu] 2 73871 serve ftp: Add ftp server (Antoine GIRARD) 73872 .IP \[bu] 2 73873 settier: perform storage tier changes on supported remotes (sandeepkru) 73874 .RE 73875 .IP \[bu] 2 73876 New Features 73877 .RS 2 73878 .IP \[bu] 2 73879 Reworked command line help 73880 .RS 2 73881 .IP \[bu] 2 73882 Make default help less verbose (Nick Craig-Wood) 73883 .IP \[bu] 2 73884 Split flags up into global and backend flags (Nick Craig-Wood) 73885 .IP \[bu] 2 73886 Implement specialised help for flags and backends (Nick Craig-Wood) 73887 .IP \[bu] 2 73888 Show URL of backend help page when starting config (Nick Craig-Wood) 73889 .RE 73890 .IP \[bu] 2 73891 stats: Long names now split in center (Joanna Marek) 73892 .IP \[bu] 2 73893 Add \f[C]--log-format\f[R] flag for more control over log output (dcpu) 73894 .IP \[bu] 2 73895 rc: Add support for OPTIONS and basic CORS (frenos) 73896 .IP \[bu] 2 73897 stats: show FatalErrors and NoRetryErrors in stats (C\['e]dric Connes) 73898 .RE 73899 .IP \[bu] 2 73900 Bug Fixes 73901 .RS 2 73902 .IP \[bu] 2 73903 Fix -P not ending with a new line (Nick Craig-Wood) 73904 .IP \[bu] 2 73905 config: don\[aq]t create default config dir when user supplies 73906 \f[C]--config\f[R] (albertony) 73907 .IP \[bu] 2 73908 Don\[aq]t print non-ASCII characters with \f[C]--progress\f[R] on 73909 windows (Nick Craig-Wood) 73910 .IP \[bu] 2 73911 Correct logs for excluded items (ssaqua) 73912 .RE 73913 .IP \[bu] 2 73914 Mount 73915 .RS 2 73916 .IP \[bu] 2 73917 Remove EXPERIMENTAL tags (Nick Craig-Wood) 73918 .RE 73919 .IP \[bu] 2 73920 VFS 73921 .RS 2 73922 .IP \[bu] 2 73923 Fix race condition detected by serve ftp tests (Nick Craig-Wood) 73924 .IP \[bu] 2 73925 Add vfs/poll-interval rc command (Fabian M\[:o]ller) 73926 .IP \[bu] 2 73927 Enable rename for nearly all remotes using server-side Move or Copy 73928 (Nick Craig-Wood) 73929 .IP \[bu] 2 73930 Reduce directory cache cleared by poll-interval (Fabian M\[:o]ller) 73931 .IP \[bu] 2 73932 Remove EXPERIMENTAL tags (Nick Craig-Wood) 73933 .RE 73934 .IP \[bu] 2 73935 Local 73936 .RS 2 73937 .IP \[bu] 2 73938 Skip bad symlinks in dir listing with -L enabled (C\['e]dric Connes) 73939 .IP \[bu] 2 73940 Preallocate files on Windows to reduce fragmentation (Nick Craig-Wood) 73941 .IP \[bu] 2 73942 Preallocate files on linux with fallocate(2) (Nick Craig-Wood) 73943 .RE 73944 .IP \[bu] 2 73945 Cache 73946 .RS 2 73947 .IP \[bu] 2 73948 Add cache/fetch rc function (Fabian M\[:o]ller) 73949 .IP \[bu] 2 73950 Fix worker scale down (Fabian M\[:o]ller) 73951 .IP \[bu] 2 73952 Improve performance by not sending info requests for cached chunks 73953 (dcpu) 73954 .IP \[bu] 2 73955 Fix error return value of cache/fetch rc method (Fabian M\[:o]ller) 73956 .IP \[bu] 2 73957 Documentation fix for cache-chunk-total-size (Anagh Kumar Baranwal) 73958 .IP \[bu] 2 73959 Preserve leading / in wrapped remote path (Fabian M\[:o]ller) 73960 .IP \[bu] 2 73961 Add plex_insecure option to skip certificate validation (Fabian 73962 M\[:o]ller) 73963 .IP \[bu] 2 73964 Remove entries that no longer exist in the source (dcpu) 73965 .RE 73966 .IP \[bu] 2 73967 Crypt 73968 .RS 2 73969 .IP \[bu] 2 73970 Preserve leading / in wrapped remote path (Fabian M\[:o]ller) 73971 .RE 73972 .IP \[bu] 2 73973 Alias 73974 .RS 2 73975 .IP \[bu] 2 73976 Fix handling of Windows network paths (Nick Craig-Wood) 73977 .RE 73978 .IP \[bu] 2 73979 Azure Blob 73980 .RS 2 73981 .IP \[bu] 2 73982 Add \f[C]--azureblob-list-chunk\f[R] parameter (Santiago Rodr\['i]guez) 73983 .IP \[bu] 2 73984 Implemented settier command support on azureblob remote. 73985 (sandeepkru) 73986 .IP \[bu] 2 73987 Work around SDK bug which causes errors for chunk-sized files (Nick 73988 Craig-Wood) 73989 .RE 73990 .IP \[bu] 2 73991 Box 73992 .RS 2 73993 .IP \[bu] 2 73994 Implement link sharing. 73995 (Sebastian B\[:u]nger) 73996 .RE 73997 .IP \[bu] 2 73998 Drive 73999 .RS 2 74000 .IP \[bu] 2 74001 Add \f[C]--drive-import-formats\f[R] - google docs can now be imported 74002 (Fabian M\[:o]ller) 74003 .RS 2 74004 .IP \[bu] 2 74005 Rewrite mime type and extension handling (Fabian M\[:o]ller) 74006 .IP \[bu] 2 74007 Add document links (Fabian M\[:o]ller) 74008 .IP \[bu] 2 74009 Add support for multipart document extensions (Fabian M\[:o]ller) 74010 .IP \[bu] 2 74011 Add support for apps-script to json export (Fabian M\[:o]ller) 74012 .IP \[bu] 2 74013 Fix escaped chars in documents during list (Fabian M\[:o]ller) 74014 .RE 74015 .IP \[bu] 2 74016 Add \f[C]--drive-v2-download-min-size\f[R] a workaround for slow 74017 downloads (Fabian M\[:o]ller) 74018 .IP \[bu] 2 74019 Improve directory notifications in ChangeNotify (Fabian M\[:o]ller) 74020 .IP \[bu] 2 74021 When listing team drives in config, continue on failure (Nick 74022 Craig-Wood) 74023 .RE 74024 .IP \[bu] 2 74025 FTP 74026 .RS 2 74027 .IP \[bu] 2 74028 Add a small pause after failed upload before deleting file (Nick 74029 Craig-Wood) 74030 .RE 74031 .IP \[bu] 2 74032 Google Cloud Storage 74033 .RS 2 74034 .IP \[bu] 2 74035 Fix service_account_file being ignored (Fabian M\[:o]ller) 74036 .RE 74037 .IP \[bu] 2 74038 Jottacloud 74039 .RS 2 74040 .IP \[bu] 2 74041 Minor improvement in quota info (omit if unlimited) (albertony) 74042 .IP \[bu] 2 74043 Add \f[C]--fast-list\f[R] support (albertony) 74044 .IP \[bu] 2 74045 Add permanent delete support: \f[C]--jottacloud-hard-delete\f[R] 74046 (albertony) 74047 .IP \[bu] 2 74048 Add link sharing support (albertony) 74049 .IP \[bu] 2 74050 Fix handling of reserved characters. 74051 (Sebastian B\[:u]nger) 74052 .IP \[bu] 2 74053 Fix socket leak on Object.Remove (Nick Craig-Wood) 74054 .RE 74055 .IP \[bu] 2 74056 Onedrive 74057 .RS 2 74058 .IP \[bu] 2 74059 Rework to support Microsoft Graph (Cnly) 74060 .RS 2 74061 .IP \[bu] 2 74062 \f[B]NB\f[R] this will require re-authenticating the remote 74063 .RE 74064 .IP \[bu] 2 74065 Removed upload cutoff and always do session uploads (Oliver Heyme) 74066 .IP \[bu] 2 74067 Use single-part upload for empty files (Cnly) 74068 .IP \[bu] 2 74069 Fix new fields not saved when editing old config (Alex Chen) 74070 .IP \[bu] 2 74071 Fix sometimes special chars in filenames not replaced (Alex Chen) 74072 .IP \[bu] 2 74073 Ignore OneNote files by default (Alex Chen) 74074 .IP \[bu] 2 74075 Add link sharing support (jackyzy823) 74076 .RE 74077 .IP \[bu] 2 74078 S3 74079 .RS 2 74080 .IP \[bu] 2 74081 Use custom pacer, to retry operations when reasonable (Craig Miskell) 74082 .IP \[bu] 2 74083 Use configured server-side-encryption and storage class options when 74084 calling CopyObject() (Paul Kohout) 74085 .IP \[bu] 2 74086 Make \f[C]--s3-v2-auth\f[R] flag (Nick Craig-Wood) 74087 .IP \[bu] 2 74088 Fix v2 auth on files with spaces (Nick Craig-Wood) 74089 .RE 74090 .IP \[bu] 2 74091 Union 74092 .RS 2 74093 .IP \[bu] 2 74094 Implement union backend which reads from multiple backends (Felix 74095 Brucker) 74096 .IP \[bu] 2 74097 Implement optional interfaces (Move, DirMove, Copy, etc.) (Nick 74098 Craig-Wood) 74099 .IP \[bu] 2 74100 Fix ChangeNotify to support multiple remotes (Fabian M\[:o]ller) 74101 .IP \[bu] 2 74102 Fix \f[C]--backup-dir\f[R] on union backend (Nick Craig-Wood) 74103 .RE 74104 .IP \[bu] 2 74105 WebDAV 74106 .RS 2 74107 .IP \[bu] 2 74108 Add another time format (Nick Craig-Wood) 74109 .IP \[bu] 2 74110 Add a small pause after failed upload before deleting file (Nick 74111 Craig-Wood) 74112 .IP \[bu] 2 74113 Add workaround for missing mtime (buergi) 74114 .IP \[bu] 2 74115 Sharepoint: Renew cookies after 12hrs (Henning Surmeier) 74116 .RE 74117 .IP \[bu] 2 74118 Yandex 74119 .RS 2 74120 .IP \[bu] 2 74121 Remove redundant nil checks (teresy) 74122 .RE 74123 .SS v1.43.1 - 2018-09-07 74124 .PP 74125 Point release to fix hubic and azureblob backends. 74126 .IP \[bu] 2 74127 Bug Fixes 74128 .RS 2 74129 .IP \[bu] 2 74130 ncdu: Return error instead of log.Fatal in Show (Fabian M\[:o]ller) 74131 .IP \[bu] 2 74132 cmd: Fix crash with \f[C]--progress\f[R] and \f[C]--stats 0\f[R] (Nick 74133 Craig-Wood) 74134 .IP \[bu] 2 74135 docs: Tidy website display (Anagh Kumar Baranwal) 74136 .RE 74137 .IP \[bu] 2 74138 Azure Blob: 74139 .RS 2 74140 .IP \[bu] 2 74141 Fix multi-part uploads. 74142 (sandeepkru) 74143 .RE 74144 .IP \[bu] 2 74145 Hubic 74146 .RS 2 74147 .IP \[bu] 2 74148 Fix uploads (Nick Craig-Wood) 74149 .IP \[bu] 2 74150 Retry auth fetching if it fails to make hubic more reliable (Nick 74151 Craig-Wood) 74152 .RE 74153 .SS v1.43 - 2018-09-01 74154 .IP \[bu] 2 74155 New backends 74156 .RS 2 74157 .IP \[bu] 2 74158 Jottacloud (Sebastian B\[:u]nger) 74159 .RE 74160 .IP \[bu] 2 74161 New commands 74162 .RS 2 74163 .IP \[bu] 2 74164 copyurl: copies a URL to a remote (Denis) 74165 .RE 74166 .IP \[bu] 2 74167 New Features 74168 .RS 2 74169 .IP \[bu] 2 74170 Reworked config for backends (Nick Craig-Wood) 74171 .RS 2 74172 .IP \[bu] 2 74173 All backend config can now be supplied by command line, env var or 74174 config file 74175 .IP \[bu] 2 74176 Advanced section in the config wizard for the optional items 74177 .IP \[bu] 2 74178 A large step towards rclone backends being usable in other go software 74179 .IP \[bu] 2 74180 Allow on the fly remotes with :backend: syntax 74181 .RE 74182 .IP \[bu] 2 74183 Stats revamp 74184 .RS 2 74185 .IP \[bu] 2 74186 Add \f[C]--progress\f[R]/\f[C]-P\f[R] flag to show interactive progress 74187 (Nick Craig-Wood) 74188 .IP \[bu] 2 74189 Show the total progress of the sync in the stats (Nick Craig-Wood) 74190 .IP \[bu] 2 74191 Add \f[C]--stats-one-line\f[R] flag for single line stats (Nick 74192 Craig-Wood) 74193 .RE 74194 .IP \[bu] 2 74195 Added weekday schedule into \f[C]--bwlimit\f[R] (Mateusz) 74196 .IP \[bu] 2 74197 lsjson: Add option to show the original object IDs (Fabian M\[:o]ller) 74198 .IP \[bu] 2 74199 serve webdav: Make Content-Type without reading the file and add 74200 \f[C]--etag-hash\f[R] (Nick Craig-Wood) 74201 .IP \[bu] 2 74202 build 74203 .RS 2 74204 .IP \[bu] 2 74205 Build macOS with native compiler (Nick Craig-Wood) 74206 .IP \[bu] 2 74207 Update to use go1.11 for the build (Nick Craig-Wood) 74208 .RE 74209 .IP \[bu] 2 74210 rc 74211 .RS 2 74212 .IP \[bu] 2 74213 Added core/stats to return the stats (reddi1) 74214 .RE 74215 .IP \[bu] 2 74216 \f[C]version --check\f[R]: Prints the current release and beta versions 74217 (Nick Craig-Wood) 74218 .RE 74219 .IP \[bu] 2 74220 Bug Fixes 74221 .RS 2 74222 .IP \[bu] 2 74223 accounting 74224 .RS 2 74225 .IP \[bu] 2 74226 Fix time to completion estimates (Nick Craig-Wood) 74227 .IP \[bu] 2 74228 Fix moving average speed for file stats (Nick Craig-Wood) 74229 .RE 74230 .IP \[bu] 2 74231 config: Fix error reading password from piped input (Nick Craig-Wood) 74232 .IP \[bu] 2 74233 move: Fix \f[C]--delete-empty-src-dirs\f[R] flag to delete all empty 74234 dirs on move (ishuah) 74235 .RE 74236 .IP \[bu] 2 74237 Mount 74238 .RS 2 74239 .IP \[bu] 2 74240 Implement \f[C]--daemon-timeout\f[R] flag for OSXFUSE (Nick Craig-Wood) 74241 .IP \[bu] 2 74242 Fix mount \f[C]--daemon\f[R] not working with encrypted config (Alex 74243 Chen) 74244 .IP \[bu] 2 74245 Clip the number of blocks to 2\[ha]32-1 on macOS - fixes borg backup 74246 (Nick Craig-Wood) 74247 .RE 74248 .IP \[bu] 2 74249 VFS 74250 .RS 2 74251 .IP \[bu] 2 74252 Enable vfs-read-chunk-size by default (Fabian M\[:o]ller) 74253 .IP \[bu] 2 74254 Add the vfs/refresh rc command (Fabian M\[:o]ller) 74255 .IP \[bu] 2 74256 Add non recursive mode to vfs/refresh rc command (Fabian M\[:o]ller) 74257 .IP \[bu] 2 74258 Try to seek buffer on read only files (Fabian M\[:o]ller) 74259 .RE 74260 .IP \[bu] 2 74261 Local 74262 .RS 2 74263 .IP \[bu] 2 74264 Fix crash when deprecated \f[C]--local-no-unicode-normalization\f[R] is 74265 supplied (Nick Craig-Wood) 74266 .IP \[bu] 2 74267 Fix mkdir error when trying to copy files to the root of a drive on 74268 windows (Nick Craig-Wood) 74269 .RE 74270 .IP \[bu] 2 74271 Cache 74272 .RS 2 74273 .IP \[bu] 2 74274 Fix nil pointer deref when using lsjson on cached directory (Nick 74275 Craig-Wood) 74276 .IP \[bu] 2 74277 Fix nil pointer deref for occasional crash on playback (Nick Craig-Wood) 74278 .RE 74279 .IP \[bu] 2 74280 Crypt 74281 .RS 2 74282 .IP \[bu] 2 74283 Fix accounting when checking hashes on upload (Nick Craig-Wood) 74284 .RE 74285 .IP \[bu] 2 74286 Amazon Cloud Drive 74287 .RS 2 74288 .IP \[bu] 2 74289 Make very clear in the docs that rclone has no ACD keys (Nick 74290 Craig-Wood) 74291 .RE 74292 .IP \[bu] 2 74293 Azure Blob 74294 .RS 2 74295 .IP \[bu] 2 74296 Add connection string and SAS URL auth (Nick Craig-Wood) 74297 .IP \[bu] 2 74298 List the container to see if it exists (Nick Craig-Wood) 74299 .IP \[bu] 2 74300 Port new Azure Blob Storage SDK (sandeepkru) 74301 .IP \[bu] 2 74302 Added blob tier, tier between Hot, Cool and Archive. 74303 (sandeepkru) 74304 .IP \[bu] 2 74305 Remove leading / from paths (Nick Craig-Wood) 74306 .RE 74307 .IP \[bu] 2 74308 B2 74309 .RS 2 74310 .IP \[bu] 2 74311 Support Application Keys (Nick Craig-Wood) 74312 .IP \[bu] 2 74313 Remove leading / from paths (Nick Craig-Wood) 74314 .RE 74315 .IP \[bu] 2 74316 Box 74317 .RS 2 74318 .IP \[bu] 2 74319 Fix upload of > 2GB files on 32 bit platforms (Nick Craig-Wood) 74320 .IP \[bu] 2 74321 Make \f[C]--box-commit-retries\f[R] flag defaulting to 100 to fix large 74322 uploads (Nick Craig-Wood) 74323 .RE 74324 .IP \[bu] 2 74325 Drive 74326 .RS 2 74327 .IP \[bu] 2 74328 Add \f[C]--drive-keep-revision-forever\f[R] flag (lewapm) 74329 .IP \[bu] 2 74330 Handle gdocs when filtering file names in list (Fabian M\[:o]ller) 74331 .IP \[bu] 2 74332 Support using \f[C]--fast-list\f[R] for large speedups (Fabian 74333 M\[:o]ller) 74334 .RE 74335 .IP \[bu] 2 74336 FTP 74337 .RS 2 74338 .IP \[bu] 2 74339 Fix Put mkParentDir failed: 521 for BunnyCDN (Nick Craig-Wood) 74340 .RE 74341 .IP \[bu] 2 74342 Google Cloud Storage 74343 .RS 2 74344 .IP \[bu] 2 74345 Fix index out of range error with \f[C]--fast-list\f[R] (Nick 74346 Craig-Wood) 74347 .RE 74348 .IP \[bu] 2 74349 Jottacloud 74350 .RS 2 74351 .IP \[bu] 2 74352 Fix MD5 error check (Oliver Heyme) 74353 .IP \[bu] 2 74354 Handle empty time values (Martin Polden) 74355 .IP \[bu] 2 74356 Calculate missing MD5s (Oliver Heyme) 74357 .IP \[bu] 2 74358 Docs, fixes and tests for MD5 calculation (Nick Craig-Wood) 74359 .IP \[bu] 2 74360 Add optional MimeTyper interface. 74361 (Sebastian B\[:u]nger) 74362 .IP \[bu] 2 74363 Implement optional About interface (for \f[C]df\f[R] support). 74364 (Sebastian B\[:u]nger) 74365 .RE 74366 .IP \[bu] 2 74367 Mega 74368 .RS 2 74369 .IP \[bu] 2 74370 Wait for events instead of arbitrary sleeping (Nick Craig-Wood) 74371 .IP \[bu] 2 74372 Add \f[C]--mega-hard-delete\f[R] flag (Nick Craig-Wood) 74373 .IP \[bu] 2 74374 Fix failed logins with upper case chars in email (Nick Craig-Wood) 74375 .RE 74376 .IP \[bu] 2 74377 Onedrive 74378 .RS 2 74379 .IP \[bu] 2 74380 Shared folder support (Yoni Jah) 74381 .IP \[bu] 2 74382 Implement DirMove (Cnly) 74383 .IP \[bu] 2 74384 Fix rmdir sometimes deleting directories with contents (Nick Craig-Wood) 74385 .RE 74386 .IP \[bu] 2 74387 Pcloud 74388 .RS 2 74389 .IP \[bu] 2 74390 Delete half uploaded files on upload error (Nick Craig-Wood) 74391 .RE 74392 .IP \[bu] 2 74393 Qingstor 74394 .RS 2 74395 .IP \[bu] 2 74396 Remove leading / from paths (Nick Craig-Wood) 74397 .RE 74398 .IP \[bu] 2 74399 S3 74400 .RS 2 74401 .IP \[bu] 2 74402 Fix index out of range error with \f[C]--fast-list\f[R] (Nick 74403 Craig-Wood) 74404 .IP \[bu] 2 74405 Add \f[C]--s3-force-path-style\f[R] (Nick Craig-Wood) 74406 .IP \[bu] 2 74407 Add support for KMS Key ID (bsteiss) 74408 .IP \[bu] 2 74409 Remove leading / from paths (Nick Craig-Wood) 74410 .RE 74411 .IP \[bu] 2 74412 Swift 74413 .RS 2 74414 .IP \[bu] 2 74415 Add \f[C]storage_policy\f[R] (Ruben Vandamme) 74416 .IP \[bu] 2 74417 Make it so just \f[C]storage_url\f[R] or \f[C]auth_token\f[R] can be 74418 overridden (Nick Craig-Wood) 74419 .IP \[bu] 2 74420 Fix server-side copy bug for unusual file names (Nick Craig-Wood) 74421 .IP \[bu] 2 74422 Remove leading / from paths (Nick Craig-Wood) 74423 .RE 74424 .IP \[bu] 2 74425 WebDAV 74426 .RS 2 74427 .IP \[bu] 2 74428 Ensure we call MKCOL with a URL with a trailing / for QNAP interop (Nick 74429 Craig-Wood) 74430 .IP \[bu] 2 74431 If root ends with / then don\[aq]t check if it is a file (Nick 74432 Craig-Wood) 74433 .IP \[bu] 2 74434 Don\[aq]t accept redirects when reading metadata (Nick Craig-Wood) 74435 .IP \[bu] 2 74436 Add bearer token (Macaroon) support for dCache (Nick Craig-Wood) 74437 .IP \[bu] 2 74438 Document dCache and Macaroons (Onno Zweers) 74439 .IP \[bu] 2 74440 Sharepoint recursion with different depth (Henning) 74441 .IP \[bu] 2 74442 Attempt to remove failed uploads (Nick Craig-Wood) 74443 .RE 74444 .IP \[bu] 2 74445 Yandex 74446 .RS 2 74447 .IP \[bu] 2 74448 Fix listing/deleting files in the root (Nick Craig-Wood) 74449 .RE 74450 .SS v1.42 - 2018-06-16 74451 .IP \[bu] 2 74452 New backends 74453 .RS 2 74454 .IP \[bu] 2 74455 OpenDrive (Oliver Heyme, Jakub Karlicek, ncw) 74456 .RE 74457 .IP \[bu] 2 74458 New commands 74459 .RS 2 74460 .IP \[bu] 2 74461 deletefile command (Filip Bartodziej) 74462 .RE 74463 .IP \[bu] 2 74464 New Features 74465 .RS 2 74466 .IP \[bu] 2 74467 copy, move: Copy single files directly, don\[aq]t use 74468 \f[C]--files-from\f[R] work-around 74469 .RS 2 74470 .IP \[bu] 2 74471 this makes them much more efficient 74472 .RE 74473 .IP \[bu] 2 74474 Implement \f[C]--max-transfer\f[R] flag to quit transferring at a limit 74475 .RS 2 74476 .IP \[bu] 2 74477 make exit code 8 for \f[C]--max-transfer\f[R] exceeded 74478 .RE 74479 .IP \[bu] 2 74480 copy: copy empty source directories to destination (Ishuah Kariuki) 74481 .IP \[bu] 2 74482 check: Add \f[C]--one-way\f[R] flag (Kasper Byrdal Nielsen) 74483 .IP \[bu] 2 74484 Add siginfo handler for macOS for ctrl-T stats (kubatasiemski) 74485 .IP \[bu] 2 74486 rc 74487 .RS 2 74488 .IP \[bu] 2 74489 add core/gc to run a garbage collection on demand 74490 .IP \[bu] 2 74491 enable go profiling by default on the \f[C]--rc\f[R] port 74492 .IP \[bu] 2 74493 return error from remote on failure 74494 .RE 74495 .IP \[bu] 2 74496 lsf 74497 .RS 2 74498 .IP \[bu] 2 74499 Add \f[C]--absolute\f[R] flag to add a leading / onto path names 74500 .IP \[bu] 2 74501 Add \f[C]--csv\f[R] flag for compliant CSV output 74502 .IP \[bu] 2 74503 Add \[aq]m\[aq] format specifier to show the MimeType 74504 .IP \[bu] 2 74505 Implement \[aq]i\[aq] format for showing object ID 74506 .RE 74507 .IP \[bu] 2 74508 lsjson 74509 .RS 2 74510 .IP \[bu] 2 74511 Add MimeType to the output 74512 .IP \[bu] 2 74513 Add ID field to output to show Object ID 74514 .RE 74515 .IP \[bu] 2 74516 Add \f[C]--retries-sleep\f[R] flag (Benjamin Joseph Dag) 74517 .IP \[bu] 2 74518 Oauth tidy up web page and error handling (Henning Surmeier) 74519 .RE 74520 .IP \[bu] 2 74521 Bug Fixes 74522 .RS 2 74523 .IP \[bu] 2 74524 Password prompt output with \f[C]--log-file\f[R] fixed for unix (Filip 74525 Bartodziej) 74526 .IP \[bu] 2 74527 Calculate ModifyWindow each time on the fly to fix various problems 74528 (Stefan Breunig) 74529 .RE 74530 .IP \[bu] 2 74531 Mount 74532 .RS 2 74533 .IP \[bu] 2 74534 Only print \[dq]File.rename error\[dq] if there actually is an error 74535 (Stefan Breunig) 74536 .IP \[bu] 2 74537 Delay rename if file has open writers instead of failing outright 74538 (Stefan Breunig) 74539 .IP \[bu] 2 74540 Ensure atexit gets run on interrupt 74541 .IP \[bu] 2 74542 macOS enhancements 74543 .RS 2 74544 .IP \[bu] 2 74545 Make \f[C]--noappledouble\f[R] \f[C]--noapplexattr\f[R] 74546 .IP \[bu] 2 74547 Add \f[C]--volname\f[R] flag and remove special chars from it 74548 .IP \[bu] 2 74549 Make Get/List/Set/Remove xattr return ENOSYS for efficiency 74550 .IP \[bu] 2 74551 Make \f[C]--daemon\f[R] work for macOS without CGO 74552 .RE 74553 .RE 74554 .IP \[bu] 2 74555 VFS 74556 .RS 2 74557 .IP \[bu] 2 74558 Add \f[C]--vfs-read-chunk-size\f[R] and 74559 \f[C]--vfs-read-chunk-size-limit\f[R] (Fabian M\[:o]ller) 74560 .IP \[bu] 2 74561 Fix ChangeNotify for new or changed folders (Fabian M\[:o]ller) 74562 .RE 74563 .IP \[bu] 2 74564 Local 74565 .RS 2 74566 .IP \[bu] 2 74567 Fix symlink/junction point directory handling under Windows 74568 .RS 2 74569 .IP \[bu] 2 74570 \f[B]NB\f[R] you will need to add \f[C]-L\f[R] to your command line to 74571 copy files with reparse points 74572 .RE 74573 .RE 74574 .IP \[bu] 2 74575 Cache 74576 .RS 2 74577 .IP \[bu] 2 74578 Add non cached dirs on notifications (Remus Bunduc) 74579 .IP \[bu] 2 74580 Allow root to be expired from rc (Remus Bunduc) 74581 .IP \[bu] 2 74582 Clean remaining empty folders from temp upload path (Remus Bunduc) 74583 .IP \[bu] 2 74584 Cache lists using batch writes (Remus Bunduc) 74585 .IP \[bu] 2 74586 Use secure websockets for HTTPS Plex addresses (John Clayton) 74587 .IP \[bu] 2 74588 Reconnect plex websocket on failures (Remus Bunduc) 74589 .IP \[bu] 2 74590 Fix panic when running without plex configs (Remus Bunduc) 74591 .IP \[bu] 2 74592 Fix root folder caching (Remus Bunduc) 74593 .RE 74594 .IP \[bu] 2 74595 Crypt 74596 .RS 2 74597 .IP \[bu] 2 74598 Check the encrypted hash of files when uploading for extra data security 74599 .RE 74600 .IP \[bu] 2 74601 Dropbox 74602 .RS 2 74603 .IP \[bu] 2 74604 Make Dropbox for business folders accessible using an initial 74605 \f[C]/\f[R] in the path 74606 .RE 74607 .IP \[bu] 2 74608 Google Cloud Storage 74609 .RS 2 74610 .IP \[bu] 2 74611 Low level retry all operations if necessary 74612 .RE 74613 .IP \[bu] 2 74614 Google Drive 74615 .RS 2 74616 .IP \[bu] 2 74617 Add \f[C]--drive-acknowledge-abuse\f[R] to download flagged files 74618 .IP \[bu] 2 74619 Add \f[C]--drive-alternate-export\f[R] to fix large doc export 74620 .IP \[bu] 2 74621 Don\[aq]t attempt to choose Team Drives when using rclone config create 74622 .IP \[bu] 2 74623 Fix change list polling with team drives 74624 .IP \[bu] 2 74625 Fix ChangeNotify for folders (Fabian M\[:o]ller) 74626 .IP \[bu] 2 74627 Fix about (and df on a mount) for team drives 74628 .RE 74629 .IP \[bu] 2 74630 Onedrive 74631 .RS 2 74632 .IP \[bu] 2 74633 Errorhandler for onedrive for business requests (Henning Surmeier) 74634 .RE 74635 .IP \[bu] 2 74636 S3 74637 .RS 2 74638 .IP \[bu] 2 74639 Adjust upload concurrency with \f[C]--s3-upload-concurrency\f[R] 74640 (themylogin) 74641 .IP \[bu] 2 74642 Fix \f[C]--s3-chunk-size\f[R] which was always using the minimum 74643 .RE 74644 .IP \[bu] 2 74645 SFTP 74646 .RS 2 74647 .IP \[bu] 2 74648 Add \f[C]--ssh-path-override\f[R] flag (Piotr Oleszczyk) 74649 .IP \[bu] 2 74650 Fix slow downloads for long latency connections 74651 .RE 74652 .IP \[bu] 2 74653 Webdav 74654 .RS 2 74655 .IP \[bu] 2 74656 Add workarounds for biz.mail.ru 74657 .IP \[bu] 2 74658 Ignore Reason-Phrase in status line to fix 4shared (Rodrigo) 74659 .IP \[bu] 2 74660 Better error message generation 74661 .RE 74662 .SS v1.41 - 2018-04-28 74663 .IP \[bu] 2 74664 New backends 74665 .RS 2 74666 .IP \[bu] 2 74667 Mega support added 74668 .IP \[bu] 2 74669 Webdav now supports SharePoint cookie authentication (hensur) 74670 .RE 74671 .IP \[bu] 2 74672 New commands 74673 .RS 2 74674 .IP \[bu] 2 74675 link: create public link to files and folders (Stefan Breunig) 74676 .IP \[bu] 2 74677 about: gets quota info from a remote (a-roussos, ncw) 74678 .IP \[bu] 2 74679 hashsum: a generic tool for any hash to produce md5sum like output 74680 .RE 74681 .IP \[bu] 2 74682 New Features 74683 .RS 2 74684 .IP \[bu] 2 74685 lsd: Add -R flag and fix and update docs for all ls commands 74686 .IP \[bu] 2 74687 ncdu: added a \[dq]refresh\[dq] key - CTRL-L (Keith Goldfarb) 74688 .IP \[bu] 2 74689 serve restic: Add append-only mode (Steve Kriss) 74690 .IP \[bu] 2 74691 serve restic: Disallow overwriting files in append-only mode (Alexander 74692 Neumann) 74693 .IP \[bu] 2 74694 serve restic: Print actual listener address (Matt Holt) 74695 .IP \[bu] 2 74696 size: Add --json flag (Matthew Holt) 74697 .IP \[bu] 2 74698 sync: implement --ignore-errors (Mateusz Pabian) 74699 .IP \[bu] 2 74700 dedupe: Add dedupe largest functionality (Richard Yang) 74701 .IP \[bu] 2 74702 fs: Extend SizeSuffix to include TB and PB for rclone about 74703 .IP \[bu] 2 74704 fs: add --dump goroutines and --dump openfiles for debugging 74705 .IP \[bu] 2 74706 rc: implement core/memstats to print internal memory usage info 74707 .IP \[bu] 2 74708 rc: new call rc/pid (Michael P. 74709 Dubner) 74710 .RE 74711 .IP \[bu] 2 74712 Compile 74713 .RS 2 74714 .IP \[bu] 2 74715 Drop support for go1.6 74716 .RE 74717 .IP \[bu] 2 74718 Release 74719 .RS 2 74720 .IP \[bu] 2 74721 Fix \f[C]make tarball\f[R] (Chih-Hsuan Yen) 74722 .RE 74723 .IP \[bu] 2 74724 Bug Fixes 74725 .RS 2 74726 .IP \[bu] 2 74727 filter: fix --min-age and --max-age together check 74728 .IP \[bu] 2 74729 fs: limit MaxIdleConns and MaxIdleConnsPerHost in transport 74730 .IP \[bu] 2 74731 lsd,lsf: make sure all times we output are in local time 74732 .IP \[bu] 2 74733 rc: fix setting bwlimit to unlimited 74734 .IP \[bu] 2 74735 rc: take note of the --rc-addr flag too as per the docs 74736 .RE 74737 .IP \[bu] 2 74738 Mount 74739 .RS 2 74740 .IP \[bu] 2 74741 Use About to return the correct disk total/used/free (e.g. 74742 in \f[C]df\f[R]) 74743 .IP \[bu] 2 74744 Set \f[C]--attr-timeout default\f[R] to \f[C]1s\f[R] - fixes: 74745 .RS 2 74746 .IP \[bu] 2 74747 rclone using too much memory 74748 .IP \[bu] 2 74749 rclone not serving files to samba 74750 .IP \[bu] 2 74751 excessive time listing directories 74752 .RE 74753 .IP \[bu] 2 74754 Fix \f[C]df -i\f[R] (upstream fix) 74755 .RE 74756 .IP \[bu] 2 74757 VFS 74758 .RS 2 74759 .IP \[bu] 2 74760 Filter files \f[C].\f[R] and \f[C]..\f[R] from directory listing 74761 .IP \[bu] 2 74762 Only make the VFS cache if --vfs-cache-mode > Off 74763 .RE 74764 .IP \[bu] 2 74765 Local 74766 .RS 2 74767 .IP \[bu] 2 74768 Add --local-no-check-updated to disable updated file checks 74769 .IP \[bu] 2 74770 Retry remove on Windows sharing violation error 74771 .RE 74772 .IP \[bu] 2 74773 Cache 74774 .RS 2 74775 .IP \[bu] 2 74776 Flush the memory cache after close 74777 .IP \[bu] 2 74778 Purge file data on notification 74779 .IP \[bu] 2 74780 Always forget parent dir for notifications 74781 .IP \[bu] 2 74782 Integrate with Plex websocket 74783 .IP \[bu] 2 74784 Add rc cache/stats (seuffert) 74785 .IP \[bu] 2 74786 Add info log on notification 74787 .RE 74788 .IP \[bu] 2 74789 Box 74790 .RS 2 74791 .IP \[bu] 2 74792 Fix failure reading large directories - parse file/directory size as 74793 float 74794 .RE 74795 .IP \[bu] 2 74796 Dropbox 74797 .RS 2 74798 .IP \[bu] 2 74799 Fix crypt+obfuscate on dropbox 74800 .IP \[bu] 2 74801 Fix repeatedly uploading the same files 74802 .RE 74803 .IP \[bu] 2 74804 FTP 74805 .RS 2 74806 .IP \[bu] 2 74807 Work around strange response from box FTP server 74808 .IP \[bu] 2 74809 More workarounds for FTP servers to fix mkParentDir error 74810 .IP \[bu] 2 74811 Fix no error on listing nonexistent directory 74812 .RE 74813 .IP \[bu] 2 74814 Google Cloud Storage 74815 .RS 2 74816 .IP \[bu] 2 74817 Add service_account_credentials (Matt Holt) 74818 .IP \[bu] 2 74819 Detect bucket presence by listing it - minimises permissions needed 74820 .IP \[bu] 2 74821 Ignore zero length directory markers 74822 .RE 74823 .IP \[bu] 2 74824 Google Drive 74825 .RS 2 74826 .IP \[bu] 2 74827 Add service_account_credentials (Matt Holt) 74828 .IP \[bu] 2 74829 Fix directory move leaving a hardlinked directory behind 74830 .IP \[bu] 2 74831 Return proper google errors when Opening files 74832 .IP \[bu] 2 74833 When initialized with a filepath, optional features used incorrect root 74834 path (Stefan Breunig) 74835 .RE 74836 .IP \[bu] 2 74837 HTTP 74838 .RS 2 74839 .IP \[bu] 2 74840 Fix sync for servers which don\[aq]t return Content-Length in HEAD 74841 .RE 74842 .IP \[bu] 2 74843 Onedrive 74844 .RS 2 74845 .IP \[bu] 2 74846 Add QuickXorHash support for OneDrive for business 74847 .IP \[bu] 2 74848 Fix socket leak in multipart session upload 74849 .RE 74850 .IP \[bu] 2 74851 S3 74852 .RS 2 74853 .IP \[bu] 2 74854 Look in S3 named profile files for credentials 74855 .IP \[bu] 2 74856 Add \f[C]--s3-disable-checksum\f[R] to disable checksum uploading (Chris 74857 Redekop) 74858 .IP \[bu] 2 74859 Hierarchical configuration support (Giri Badanahatti) 74860 .IP \[bu] 2 74861 Add in config for all the supported S3 providers 74862 .IP \[bu] 2 74863 Add One Zone Infrequent Access storage class (Craig Rachel) 74864 .IP \[bu] 2 74865 Add --use-server-modtime support (Peter Baumgartner) 74866 .IP \[bu] 2 74867 Add --s3-chunk-size option to control multipart uploads 74868 .IP \[bu] 2 74869 Ignore zero length directory markers 74870 .RE 74871 .IP \[bu] 2 74872 SFTP 74873 .RS 2 74874 .IP \[bu] 2 74875 Update docs to match code, fix typos and clarify disable_hashcheck 74876 prompt (Michael G. 74877 Noll) 74878 .IP \[bu] 2 74879 Update docs with Synology quirks 74880 .IP \[bu] 2 74881 Fail soft with a debug on hash failure 74882 .RE 74883 .IP \[bu] 2 74884 Swift 74885 .RS 2 74886 .IP \[bu] 2 74887 Add --use-server-modtime support (Peter Baumgartner) 74888 .RE 74889 .IP \[bu] 2 74890 Webdav 74891 .RS 2 74892 .IP \[bu] 2 74893 Support SharePoint cookie authentication (hensur) 74894 .IP \[bu] 2 74895 Strip leading and trailing / off root 74896 .RE 74897 .SS v1.40 - 2018-03-19 74898 .IP \[bu] 2 74899 New backends 74900 .RS 2 74901 .IP \[bu] 2 74902 Alias backend to create aliases for existing remote names (Fabian 74903 M\[:o]ller) 74904 .RE 74905 .IP \[bu] 2 74906 New commands 74907 .RS 2 74908 .IP \[bu] 2 74909 \f[C]lsf\f[R]: list for parsing purposes (Jakub Tasiemski) 74910 .RS 2 74911 .IP \[bu] 2 74912 by default this is a simple non recursive list of files and directories 74913 .IP \[bu] 2 74914 it can be configured to add more info in an easy to parse way 74915 .RE 74916 .IP \[bu] 2 74917 \f[C]serve restic\f[R]: for serving a remote as a Restic REST endpoint 74918 .RS 2 74919 .IP \[bu] 2 74920 This enables restic to use any backends that rclone can access 74921 .IP \[bu] 2 74922 Thanks Alexander Neumann for help, patches and review 74923 .RE 74924 .IP \[bu] 2 74925 \f[C]rc\f[R]: enable the remote control of a running rclone 74926 .RS 2 74927 .IP \[bu] 2 74928 The running rclone must be started with --rc and related flags. 74929 .IP \[bu] 2 74930 Currently there is support for bwlimit, and flushing for mount and 74931 cache. 74932 .RE 74933 .RE 74934 .IP \[bu] 2 74935 New Features 74936 .RS 2 74937 .IP \[bu] 2 74938 \f[C]--max-delete\f[R] flag to add a delete threshold (Bj\[/o]rn Erik 74939 Pedersen) 74940 .IP \[bu] 2 74941 All backends now support RangeOption for ranged Open 74942 .RS 2 74943 .IP \[bu] 2 74944 \f[C]cat\f[R]: Use RangeOption for limited fetches to make more 74945 efficient 74946 .IP \[bu] 2 74947 \f[C]cryptcheck\f[R]: make reading of nonce more efficient with 74948 RangeOption 74949 .RE 74950 .IP \[bu] 2 74951 serve http/webdav/restic 74952 .RS 2 74953 .IP \[bu] 2 74954 support SSL/TLS 74955 .IP \[bu] 2 74956 add \f[C]--user\f[R] \f[C]--pass\f[R] and \f[C]--htpasswd\f[R] for 74957 authentication 74958 .RE 74959 .IP \[bu] 2 74960 \f[C]copy\f[R]/\f[C]move\f[R]: detect file size change during copy/move 74961 and abort transfer (ishuah) 74962 .IP \[bu] 2 74963 \f[C]cryptdecode\f[R]: added option to return encrypted file names. 74964 (ishuah) 74965 .IP \[bu] 2 74966 \f[C]lsjson\f[R]: add \f[C]--encrypted\f[R] to show encrypted name 74967 (Jakub Tasiemski) 74968 .IP \[bu] 2 74969 Add \f[C]--stats-file-name-length\f[R] to specify the printed file name 74970 length for stats (Will Gunn) 74971 .RE 74972 .IP \[bu] 2 74973 Compile 74974 .RS 2 74975 .IP \[bu] 2 74976 Code base was shuffled and factored 74977 .RS 2 74978 .IP \[bu] 2 74979 backends moved into a backend directory 74980 .IP \[bu] 2 74981 large packages split up 74982 .IP \[bu] 2 74983 See the CONTRIBUTING.md doc for info as to what lives where now 74984 .RE 74985 .IP \[bu] 2 74986 Update to using go1.10 as the default go version 74987 .IP \[bu] 2 74988 Implement daily full integration 74989 tests (https://pub.rclone.org/integration-tests/) 74990 .RE 74991 .IP \[bu] 2 74992 Release 74993 .RS 2 74994 .IP \[bu] 2 74995 Include a source tarball and sign it and the binaries 74996 .IP \[bu] 2 74997 Sign the git tags as part of the release process 74998 .IP \[bu] 2 74999 Add .deb and .rpm packages as part of the build 75000 .IP \[bu] 2 75001 Make a beta release for all branches on the main repo (but not pull 75002 requests) 75003 .RE 75004 .IP \[bu] 2 75005 Bug Fixes 75006 .RS 2 75007 .IP \[bu] 2 75008 config: fixes errors on nonexistent config by loading config file only 75009 on first access 75010 .IP \[bu] 2 75011 config: retry saving the config after failure (Mateusz) 75012 .IP \[bu] 2 75013 sync: when using \f[C]--backup-dir\f[R] don\[aq]t delete files if we 75014 can\[aq]t set their modtime 75015 .RS 2 75016 .IP \[bu] 2 75017 this fixes odd behaviour with Dropbox and \f[C]--backup-dir\f[R] 75018 .RE 75019 .IP \[bu] 2 75020 fshttp: fix idle timeouts for HTTP connections 75021 .IP \[bu] 2 75022 \f[C]serve http\f[R]: fix serving files with : in - fixes 75023 .IP \[bu] 2 75024 Fix \f[C]--exclude-if-present\f[R] to ignore directories which it 75025 doesn\[aq]t have permission for (Iakov Davydov) 75026 .IP \[bu] 2 75027 Make accounting work properly with crypt and b2 75028 .IP \[bu] 2 75029 remove \f[C]--no-traverse\f[R] flag because it is obsolete 75030 .RE 75031 .IP \[bu] 2 75032 Mount 75033 .RS 2 75034 .IP \[bu] 2 75035 Add \f[C]--attr-timeout\f[R] flag to control attribute caching in kernel 75036 .RS 2 75037 .IP \[bu] 2 75038 this now defaults to 0 which is correct but less efficient 75039 .IP \[bu] 2 75040 see the mount 75041 docs (https://rclone.org/commands/rclone_mount/#attribute-caching) for 75042 more info 75043 .RE 75044 .IP \[bu] 2 75045 Add \f[C]--daemon\f[R] flag to allow mount to run in the background 75046 (ishuah) 75047 .IP \[bu] 2 75048 Fix: Return ENOSYS rather than EIO on attempted link 75049 .RS 2 75050 .IP \[bu] 2 75051 This fixes FileZilla accessing an rclone mount served over sftp. 75052 .RE 75053 .IP \[bu] 2 75054 Fix setting modtime twice 75055 .IP \[bu] 2 75056 Mount tests now run on CI for Linux (mount & cmount)/Mac/Windows 75057 .IP \[bu] 2 75058 Many bugs fixed in the VFS layer - see below 75059 .RE 75060 .IP \[bu] 2 75061 VFS 75062 .RS 2 75063 .IP \[bu] 2 75064 Many fixes for \f[C]--vfs-cache-mode\f[R] writes and above 75065 .RS 2 75066 .IP \[bu] 2 75067 Update cached copy if we know it has changed (fixes stale data) 75068 .IP \[bu] 2 75069 Clean path names before using them in the cache 75070 .IP \[bu] 2 75071 Disable cache cleaner if \f[C]--vfs-cache-poll-interval=0\f[R] 75072 .IP \[bu] 2 75073 Fill and clean the cache immediately on startup 75074 .RE 75075 .IP \[bu] 2 75076 Fix Windows opening every file when it stats the file 75077 .IP \[bu] 2 75078 Fix applying modtime for an open Write Handle 75079 .IP \[bu] 2 75080 Fix creation of files when truncating 75081 .IP \[bu] 2 75082 Write 0 bytes when flushing unwritten handles to avoid race conditions 75083 in FUSE 75084 .IP \[bu] 2 75085 Downgrade \[dq]poll-interval is not supported\[dq] message to Info 75086 .IP \[bu] 2 75087 Make OpenFile and friends return EINVAL if O_RDONLY and O_TRUNC 75088 .RE 75089 .IP \[bu] 2 75090 Local 75091 .RS 2 75092 .IP \[bu] 2 75093 Downgrade \[dq]invalid cross-device link: trying copy\[dq] to debug 75094 .IP \[bu] 2 75095 Make DirMove return fs.ErrorCantDirMove to allow fallback to Copy for 75096 cross device 75097 .IP \[bu] 2 75098 Fix race conditions updating the hashes 75099 .RE 75100 .IP \[bu] 2 75101 Cache 75102 .RS 2 75103 .IP \[bu] 2 75104 Add support for polling - cache will update when remote changes on 75105 supported backends 75106 .IP \[bu] 2 75107 Reduce log level for Plex api 75108 .IP \[bu] 2 75109 Fix dir cache issue 75110 .IP \[bu] 2 75111 Implement \f[C]--cache-db-wait-time\f[R] flag 75112 .IP \[bu] 2 75113 Improve efficiency with RangeOption and RangeSeek 75114 .IP \[bu] 2 75115 Fix dirmove with temp fs enabled 75116 .IP \[bu] 2 75117 Notify vfs when using temp fs 75118 .IP \[bu] 2 75119 Offline uploading 75120 .IP \[bu] 2 75121 Remote control support for path flushing 75122 .RE 75123 .IP \[bu] 2 75124 Amazon cloud drive 75125 .RS 2 75126 .IP \[bu] 2 75127 Rclone no longer has any working keys - disable integration tests 75128 .IP \[bu] 2 75129 Implement DirChangeNotify to notify cache/vfs/mount of changes 75130 .RE 75131 .IP \[bu] 2 75132 Azureblob 75133 .RS 2 75134 .IP \[bu] 2 75135 Don\[aq]t check for bucket/container presence if listing was OK 75136 .RS 2 75137 .IP \[bu] 2 75138 this makes rclone do one less request per invocation 75139 .RE 75140 .IP \[bu] 2 75141 Improve accounting for chunked uploads 75142 .RE 75143 .IP \[bu] 2 75144 Backblaze B2 75145 .RS 2 75146 .IP \[bu] 2 75147 Don\[aq]t check for bucket/container presence if listing was OK 75148 .RS 2 75149 .IP \[bu] 2 75150 this makes rclone do one less request per invocation 75151 .RE 75152 .RE 75153 .IP \[bu] 2 75154 Box 75155 .RS 2 75156 .IP \[bu] 2 75157 Improve accounting for chunked uploads 75158 .RE 75159 .IP \[bu] 2 75160 Dropbox 75161 .RS 2 75162 .IP \[bu] 2 75163 Fix custom oauth client parameters 75164 .RE 75165 .IP \[bu] 2 75166 Google Cloud Storage 75167 .RS 2 75168 .IP \[bu] 2 75169 Don\[aq]t check for bucket/container presence if listing was OK 75170 .RS 2 75171 .IP \[bu] 2 75172 this makes rclone do one less request per invocation 75173 .RE 75174 .RE 75175 .IP \[bu] 2 75176 Google Drive 75177 .RS 2 75178 .IP \[bu] 2 75179 Migrate to api v3 (Fabian M\[:o]ller) 75180 .IP \[bu] 2 75181 Add scope configuration and root folder selection 75182 .IP \[bu] 2 75183 Add \f[C]--drive-impersonate\f[R] for service accounts 75184 .RS 2 75185 .IP \[bu] 2 75186 thanks to everyone who tested, explored and contributed docs 75187 .RE 75188 .IP \[bu] 2 75189 Add \f[C]--drive-use-created-date\f[R] to use created date as modified 75190 date (nbuchanan) 75191 .IP \[bu] 2 75192 Request the export formats only when required 75193 .RS 2 75194 .IP \[bu] 2 75195 This makes rclone quicker when there are no google docs 75196 .RE 75197 .IP \[bu] 2 75198 Fix finding paths with latin1 chars (a workaround for a drive bug) 75199 .IP \[bu] 2 75200 Fix copying of a single Google doc file 75201 .IP \[bu] 2 75202 Fix \f[C]--drive-auth-owner-only\f[R] to look in all directories 75203 .RE 75204 .IP \[bu] 2 75205 HTTP 75206 .RS 2 75207 .IP \[bu] 2 75208 Fix handling of directories with & in 75209 .RE 75210 .IP \[bu] 2 75211 Onedrive 75212 .RS 2 75213 .IP \[bu] 2 75214 Removed upload cutoff and always do session uploads 75215 .RS 2 75216 .IP \[bu] 2 75217 this stops the creation of multiple versions on business onedrive 75218 .RE 75219 .IP \[bu] 2 75220 Overwrite object size value with real size when reading file. 75221 (Victor) 75222 .RS 2 75223 .IP \[bu] 2 75224 this fixes oddities when onedrive misreports the size of images 75225 .RE 75226 .RE 75227 .IP \[bu] 2 75228 Pcloud 75229 .RS 2 75230 .IP \[bu] 2 75231 Remove unused chunked upload flag and code 75232 .RE 75233 .IP \[bu] 2 75234 Qingstor 75235 .RS 2 75236 .IP \[bu] 2 75237 Don\[aq]t check for bucket/container presence if listing was OK 75238 .RS 2 75239 .IP \[bu] 2 75240 this makes rclone do one less request per invocation 75241 .RE 75242 .RE 75243 .IP \[bu] 2 75244 S3 75245 .RS 2 75246 .IP \[bu] 2 75247 Support hashes for multipart files (Chris Redekop) 75248 .IP \[bu] 2 75249 Initial support for IBM COS (S3) (Giri Badanahatti) 75250 .IP \[bu] 2 75251 Update docs to discourage use of v2 auth with CEPH and others 75252 .IP \[bu] 2 75253 Don\[aq]t check for bucket/container presence if listing was OK 75254 .RS 2 75255 .IP \[bu] 2 75256 this makes rclone do one less request per invocation 75257 .RE 75258 .IP \[bu] 2 75259 Fix server-side copy and set modtime on files with + in 75260 .RE 75261 .IP \[bu] 2 75262 SFTP 75263 .RS 2 75264 .IP \[bu] 2 75265 Add option to disable remote hash check command execution (Jon Fautley) 75266 .IP \[bu] 2 75267 Add \f[C]--sftp-ask-password\f[R] flag to prompt for password when 75268 needed (Leo R. 75269 Lundgren) 75270 .IP \[bu] 2 75271 Add \f[C]set_modtime\f[R] configuration option 75272 .IP \[bu] 2 75273 Fix following of symlinks 75274 .IP \[bu] 2 75275 Fix reading config file outside of Fs setup 75276 .IP \[bu] 2 75277 Fix reading $USER in username fallback not $HOME 75278 .IP \[bu] 2 75279 Fix running under crontab - Use correct OS way of reading username 75280 .RE 75281 .IP \[bu] 2 75282 Swift 75283 .RS 2 75284 .IP \[bu] 2 75285 Fix refresh of authentication token 75286 .RS 2 75287 .IP \[bu] 2 75288 in v1.39 a bug was introduced which ignored new tokens - this fixes it 75289 .RE 75290 .IP \[bu] 2 75291 Fix extra HEAD transaction when uploading a new file 75292 .IP \[bu] 2 75293 Don\[aq]t check for bucket/container presence if listing was OK 75294 .RS 2 75295 .IP \[bu] 2 75296 this makes rclone do one less request per invocation 75297 .RE 75298 .RE 75299 .IP \[bu] 2 75300 Webdav 75301 .RS 2 75302 .IP \[bu] 2 75303 Add new time formats to support mydrive.ch and others 75304 .RE 75305 .SS v1.39 - 2017-12-23 75306 .IP \[bu] 2 75307 New backends 75308 .RS 2 75309 .IP \[bu] 2 75310 WebDAV 75311 .RS 2 75312 .IP \[bu] 2 75313 tested with nextcloud, owncloud, put.io and others! 75314 .RE 75315 .IP \[bu] 2 75316 Pcloud 75317 .IP \[bu] 2 75318 cache - wraps a cache around other backends (Remus Bunduc) 75319 .RS 2 75320 .IP \[bu] 2 75321 useful in combination with mount 75322 .IP \[bu] 2 75323 NB this feature is in beta so use with care 75324 .RE 75325 .RE 75326 .IP \[bu] 2 75327 New commands 75328 .RS 2 75329 .IP \[bu] 2 75330 serve command with subcommands: 75331 .RS 2 75332 .IP \[bu] 2 75333 serve webdav: this implements a webdav server for any rclone remote. 75334 .IP \[bu] 2 75335 serve http: command to serve a remote over HTTP 75336 .RE 75337 .IP \[bu] 2 75338 config: add sub commands for full config file management 75339 .RS 2 75340 .IP \[bu] 2 75341 create/delete/dump/edit/file/password/providers/show/update 75342 .RE 75343 .IP \[bu] 2 75344 touch: to create or update the timestamp of a file (Jakub Tasiemski) 75345 .RE 75346 .IP \[bu] 2 75347 New Features 75348 .RS 2 75349 .IP \[bu] 2 75350 curl install for rclone (Filip Bartodziej) 75351 .IP \[bu] 2 75352 --stats now shows percentage, size, rate and ETA in condensed form 75353 (Ishuah Kariuki) 75354 .IP \[bu] 2 75355 --exclude-if-present to exclude a directory if a file is present (Iakov 75356 Davydov) 75357 .IP \[bu] 2 75358 rmdirs: add --leave-root flag (lewapm) 75359 .IP \[bu] 2 75360 move: add --delete-empty-src-dirs flag to remove dirs after move (Ishuah 75361 Kariuki) 75362 .IP \[bu] 2 75363 Add --dump flag, introduce --dump requests, responses and remove 75364 --dump-auth, --dump-filters 75365 .RS 2 75366 .IP \[bu] 2 75367 Obscure X-Auth-Token: from headers when dumping too 75368 .RE 75369 .IP \[bu] 2 75370 Document and implement exit codes for different failure modes (Ishuah 75371 Kariuki) 75372 .RE 75373 .IP \[bu] 2 75374 Compile 75375 .IP \[bu] 2 75376 Bug Fixes 75377 .RS 2 75378 .IP \[bu] 2 75379 Retry lots more different types of errors to make multipart transfers 75380 more reliable 75381 .IP \[bu] 2 75382 Save the config before asking for a token, fixes disappearing oauth 75383 config 75384 .IP \[bu] 2 75385 Warn the user if --include and --exclude are used together (Ernest 75386 Borowski) 75387 .IP \[bu] 2 75388 Fix duplicate files (e.g. 75389 on Google drive) causing spurious copies 75390 .IP \[bu] 2 75391 Allow trailing and leading whitespace for passwords (Jason Rose) 75392 .IP \[bu] 2 75393 ncdu: fix crashes on empty directories 75394 .IP \[bu] 2 75395 rcat: fix goroutine leak 75396 .IP \[bu] 2 75397 moveto/copyto: Fix to allow copying to the same name 75398 .RE 75399 .IP \[bu] 2 75400 Mount 75401 .RS 2 75402 .IP \[bu] 2 75403 --vfs-cache mode to make writes into mounts more reliable. 75404 .RS 2 75405 .IP \[bu] 2 75406 this requires caching files on the disk (see --cache-dir) 75407 .IP \[bu] 2 75408 As this is a new feature, use with care 75409 .RE 75410 .IP \[bu] 2 75411 Use sdnotify to signal systemd the mount is ready (Fabian M\[:o]ller) 75412 .IP \[bu] 2 75413 Check if directory is not empty before mounting (Ernest Borowski) 75414 .RE 75415 .IP \[bu] 2 75416 Local 75417 .RS 2 75418 .IP \[bu] 2 75419 Add error message for cross file system moves 75420 .IP \[bu] 2 75421 Fix equality check for times 75422 .RE 75423 .IP \[bu] 2 75424 Dropbox 75425 .RS 2 75426 .IP \[bu] 2 75427 Rework multipart upload 75428 .RS 2 75429 .IP \[bu] 2 75430 buffer the chunks when uploading large files so they can be retried 75431 .IP \[bu] 2 75432 change default chunk size to 48MB now we are buffering them in memory 75433 .IP \[bu] 2 75434 retry every error after the first chunk is done successfully 75435 .RE 75436 .IP \[bu] 2 75437 Fix error when renaming directories 75438 .RE 75439 .IP \[bu] 2 75440 Swift 75441 .RS 2 75442 .IP \[bu] 2 75443 Fix crash on bad authentication 75444 .RE 75445 .IP \[bu] 2 75446 Google Drive 75447 .RS 2 75448 .IP \[bu] 2 75449 Add service account support (Tim Cooijmans) 75450 .RE 75451 .IP \[bu] 2 75452 S3 75453 .RS 2 75454 .IP \[bu] 2 75455 Make it work properly with Digital Ocean Spaces (Andrew 75456 Starr-Bochicchio) 75457 .IP \[bu] 2 75458 Fix crash if a bad listing is received 75459 .IP \[bu] 2 75460 Add support for ECS task IAM roles (David Minor) 75461 .RE 75462 .IP \[bu] 2 75463 Backblaze B2 75464 .RS 2 75465 .IP \[bu] 2 75466 Fix multipart upload retries 75467 .IP \[bu] 2 75468 Fix --hard-delete to make it work 100% of the time 75469 .RE 75470 .IP \[bu] 2 75471 Swift 75472 .RS 2 75473 .IP \[bu] 2 75474 Allow authentication with storage URL and auth key (Giovanni Pizzi) 75475 .IP \[bu] 2 75476 Add new fields for swift configuration to support IBM Bluemix Swift 75477 (Pierre Carlson) 75478 .IP \[bu] 2 75479 Add OS_TENANT_ID and OS_USER_ID to config 75480 .IP \[bu] 2 75481 Allow configs with user id instead of user name 75482 .IP \[bu] 2 75483 Check if swift segments container exists before creating (John Leach) 75484 .IP \[bu] 2 75485 Fix memory leak in swift transfers (upstream fix) 75486 .RE 75487 .IP \[bu] 2 75488 SFTP 75489 .RS 2 75490 .IP \[bu] 2 75491 Add option to enable the use of aes128-cbc cipher (Jon Fautley) 75492 .RE 75493 .IP \[bu] 2 75494 Amazon cloud drive 75495 .RS 2 75496 .IP \[bu] 2 75497 Fix download of large files failing with \[dq]Only one auth mechanism 75498 allowed\[dq] 75499 .RE 75500 .IP \[bu] 2 75501 crypt 75502 .RS 2 75503 .IP \[bu] 2 75504 Option to encrypt directory names or leave them intact 75505 .IP \[bu] 2 75506 Implement DirChangeNotify (Fabian M\[:o]ller) 75507 .RE 75508 .IP \[bu] 2 75509 onedrive 75510 .RS 2 75511 .IP \[bu] 2 75512 Add option to choose resourceURL during setup of OneDrive Business 75513 account if more than one is available for user 75514 .RE 75515 .SS v1.38 - 2017-09-30 75516 .IP \[bu] 2 75517 New backends 75518 .RS 2 75519 .IP \[bu] 2 75520 Azure Blob Storage (thanks Andrei Dragomir) 75521 .IP \[bu] 2 75522 Box 75523 .IP \[bu] 2 75524 Onedrive for Business (thanks Oliver Heyme) 75525 .IP \[bu] 2 75526 QingStor from QingCloud (thanks wuyu) 75527 .RE 75528 .IP \[bu] 2 75529 New commands 75530 .RS 2 75531 .IP \[bu] 2 75532 \f[C]rcat\f[R] - read from standard input and stream upload 75533 .IP \[bu] 2 75534 \f[C]tree\f[R] - shows a nicely formatted recursive listing 75535 .IP \[bu] 2 75536 \f[C]cryptdecode\f[R] - decode encrypted file names (thanks ishuah) 75537 .IP \[bu] 2 75538 \f[C]config show\f[R] - print the config file 75539 .IP \[bu] 2 75540 \f[C]config file\f[R] - print the config file location 75541 .RE 75542 .IP \[bu] 2 75543 New Features 75544 .RS 2 75545 .IP \[bu] 2 75546 Empty directories are deleted on \f[C]sync\f[R] 75547 .IP \[bu] 2 75548 \f[C]dedupe\f[R] - implement merging of duplicate directories 75549 .IP \[bu] 2 75550 \f[C]check\f[R] and \f[C]cryptcheck\f[R] made more consistent and use 75551 less memory 75552 .IP \[bu] 2 75553 \f[C]cleanup\f[R] for remaining remotes (thanks ishuah) 75554 .IP \[bu] 2 75555 \f[C]--immutable\f[R] for ensuring that files don\[aq]t change (thanks 75556 Jacob McNamee) 75557 .IP \[bu] 2 75558 \f[C]--user-agent\f[R] option (thanks Alex McGrath Kraak) 75559 .IP \[bu] 2 75560 \f[C]--disable\f[R] flag to disable optional features 75561 .IP \[bu] 2 75562 \f[C]--bind\f[R] flag for choosing the local addr on outgoing 75563 connections 75564 .IP \[bu] 2 75565 Support for zsh auto-completion (thanks bpicode) 75566 .IP \[bu] 2 75567 Stop normalizing file names but do a normalized compare in 75568 \f[C]sync\f[R] 75569 .RE 75570 .IP \[bu] 2 75571 Compile 75572 .RS 2 75573 .IP \[bu] 2 75574 Update to using go1.9 as the default go version 75575 .IP \[bu] 2 75576 Remove snapd build due to maintenance problems 75577 .RE 75578 .IP \[bu] 2 75579 Bug Fixes 75580 .RS 2 75581 .IP \[bu] 2 75582 Improve retriable error detection which makes multipart uploads better 75583 .IP \[bu] 2 75584 Make \f[C]check\f[R] obey \f[C]--ignore-size\f[R] 75585 .IP \[bu] 2 75586 Fix bwlimit toggle in conjunction with schedules (thanks cbruegg) 75587 .IP \[bu] 2 75588 \f[C]config\f[R] ensures newly written config is on the same mount 75589 .RE 75590 .IP \[bu] 2 75591 Local 75592 .RS 2 75593 .IP \[bu] 2 75594 Revert to copy when moving file across file system boundaries 75595 .IP \[bu] 2 75596 \f[C]--skip-links\f[R] to suppress symlink warnings (thanks Zhiming 75597 Wang) 75598 .RE 75599 .IP \[bu] 2 75600 Mount 75601 .RS 2 75602 .IP \[bu] 2 75603 Reuse \f[C]rcat\f[R] internals to support uploads from all remotes 75604 .RE 75605 .IP \[bu] 2 75606 Dropbox 75607 .RS 2 75608 .IP \[bu] 2 75609 Fix \[dq]entry doesn\[aq]t belong in directory\[dq] error 75610 .IP \[bu] 2 75611 Stop using deprecated API methods 75612 .RE 75613 .IP \[bu] 2 75614 Swift 75615 .RS 2 75616 .IP \[bu] 2 75617 Fix server-side copy to empty container with \f[C]--fast-list\f[R] 75618 .RE 75619 .IP \[bu] 2 75620 Google Drive 75621 .RS 2 75622 .IP \[bu] 2 75623 Change the default for \f[C]--drive-use-trash\f[R] to \f[C]true\f[R] 75624 .RE 75625 .IP \[bu] 2 75626 S3 75627 .RS 2 75628 .IP \[bu] 2 75629 Set session token when using STS (thanks Girish Ramakrishnan) 75630 .IP \[bu] 2 75631 Glacier docs and error messages (thanks Jan Varho) 75632 .IP \[bu] 2 75633 Read 1000 (not 1024) items in dir listings to fix Wasabi 75634 .RE 75635 .IP \[bu] 2 75636 Backblaze B2 75637 .RS 2 75638 .IP \[bu] 2 75639 Fix SHA1 mismatch when downloading files with no SHA1 75640 .IP \[bu] 2 75641 Calculate missing hashes on the fly instead of spooling 75642 .IP \[bu] 2 75643 \f[C]--b2-hard-delete\f[R] to permanently delete (not hide) files 75644 (thanks John Papandriopoulos) 75645 .RE 75646 .IP \[bu] 2 75647 Hubic 75648 .RS 2 75649 .IP \[bu] 2 75650 Fix creating containers - no longer have to use the \f[C]default\f[R] 75651 container 75652 .RE 75653 .IP \[bu] 2 75654 Swift 75655 .RS 2 75656 .IP \[bu] 2 75657 Optionally configure from a standard set of OpenStack environment vars 75658 .IP \[bu] 2 75659 Add \f[C]endpoint_type\f[R] config 75660 .RE 75661 .IP \[bu] 2 75662 Google Cloud Storage 75663 .RS 2 75664 .IP \[bu] 2 75665 Fix bucket creation to work with limited permission users 75666 .RE 75667 .IP \[bu] 2 75668 SFTP 75669 .RS 2 75670 .IP \[bu] 2 75671 Implement connection pooling for multiple ssh connections 75672 .IP \[bu] 2 75673 Limit new connections per second 75674 .IP \[bu] 2 75675 Add support for MD5 and SHA1 hashes where available (thanks Christian 75676 Br\[:u]ggemann) 75677 .RE 75678 .IP \[bu] 2 75679 HTTP 75680 .RS 2 75681 .IP \[bu] 2 75682 Fix URL encoding issues 75683 .IP \[bu] 2 75684 Fix directories with \f[C]:\f[R] in 75685 .IP \[bu] 2 75686 Fix panic with URL encoded content 75687 .RE 75688 .SS v1.37 - 2017-07-22 75689 .IP \[bu] 2 75690 New backends 75691 .RS 2 75692 .IP \[bu] 2 75693 FTP - thanks to Antonio Messina 75694 .IP \[bu] 2 75695 HTTP - thanks to Vasiliy Tolstov 75696 .RE 75697 .IP \[bu] 2 75698 New commands 75699 .RS 2 75700 .IP \[bu] 2 75701 rclone ncdu - for exploring a remote with a text based user interface. 75702 .IP \[bu] 2 75703 rclone lsjson - for listing with a machine-readable output 75704 .IP \[bu] 2 75705 rclone dbhashsum - to show Dropbox style hashes of files (local or 75706 Dropbox) 75707 .RE 75708 .IP \[bu] 2 75709 New Features 75710 .RS 2 75711 .IP \[bu] 2 75712 Implement --fast-list flag 75713 .RS 2 75714 .IP \[bu] 2 75715 This allows remotes to list recursively if they can 75716 .IP \[bu] 2 75717 This uses less transactions (important if you pay for them) 75718 .IP \[bu] 2 75719 This may or may not be quicker 75720 .IP \[bu] 2 75721 This will use more memory as it has to hold the listing in memory 75722 .IP \[bu] 2 75723 --old-sync-method deprecated - the remaining uses are covered by 75724 --fast-list 75725 .IP \[bu] 2 75726 This involved a major re-write of all the listing code 75727 .RE 75728 .IP \[bu] 2 75729 Add --tpslimit and --tpslimit-burst to limit transactions per second 75730 .RS 2 75731 .IP \[bu] 2 75732 this is useful in conjunction with \f[C]rclone mount\f[R] to limit 75733 external apps 75734 .RE 75735 .IP \[bu] 2 75736 Add --stats-log-level so can see --stats without -v 75737 .IP \[bu] 2 75738 Print password prompts to stderr - Hraban Luyat 75739 .IP \[bu] 2 75740 Warn about duplicate files when syncing 75741 .IP \[bu] 2 75742 Oauth improvements 75743 .RS 2 75744 .IP \[bu] 2 75745 allow auth_url and token_url to be set in the config file 75746 .IP \[bu] 2 75747 Print redirection URI if using own credentials. 75748 .RE 75749 .IP \[bu] 2 75750 Don\[aq]t Mkdir at the start of sync to save transactions 75751 .RE 75752 .IP \[bu] 2 75753 Compile 75754 .RS 2 75755 .IP \[bu] 2 75756 Update build to go1.8.3 75757 .IP \[bu] 2 75758 Require go1.6 for building rclone 75759 .IP \[bu] 2 75760 Compile 386 builds with \[dq]GO386=387\[dq] for maximum compatibility 75761 .RE 75762 .IP \[bu] 2 75763 Bug Fixes 75764 .RS 2 75765 .IP \[bu] 2 75766 Fix menu selection when no remotes 75767 .IP \[bu] 2 75768 Config saving reworked to not kill the file if disk gets full 75769 .IP \[bu] 2 75770 Don\[aq]t delete remote if name does not change while renaming 75771 .IP \[bu] 2 75772 moveto, copyto: report transfers and checks as per move and copy 75773 .RE 75774 .IP \[bu] 2 75775 Local 75776 .RS 2 75777 .IP \[bu] 2 75778 Add --local-no-unicode-normalization flag - Bob Potter 75779 .RE 75780 .IP \[bu] 2 75781 Mount 75782 .RS 2 75783 .IP \[bu] 2 75784 Now supported on Windows using cgofuse and WinFsp - thanks to Bill 75785 Zissimopoulos for much help 75786 .IP \[bu] 2 75787 Compare checksums on upload/download via FUSE 75788 .IP \[bu] 2 75789 Unmount when program ends with SIGINT (Ctrl+C) or SIGTERM - 75790 J\['e]r\[^o]me Vizcaino 75791 .IP \[bu] 2 75792 On read only open of file, make open pending until first read 75793 .IP \[bu] 2 75794 Make --read-only reject modify operations 75795 .IP \[bu] 2 75796 Implement ModTime via FUSE for remotes that support it 75797 .IP \[bu] 2 75798 Allow modTime to be changed even before all writers are closed 75799 .IP \[bu] 2 75800 Fix panic on renames 75801 .IP \[bu] 2 75802 Fix hang on errored upload 75803 .RE 75804 .IP \[bu] 2 75805 Crypt 75806 .RS 2 75807 .IP \[bu] 2 75808 Report the name:root as specified by the user 75809 .IP \[bu] 2 75810 Add an \[dq]obfuscate\[dq] option for filename encryption - Stephen 75811 Harris 75812 .RE 75813 .IP \[bu] 2 75814 Amazon Drive 75815 .RS 2 75816 .IP \[bu] 2 75817 Fix initialization order for token renewer 75818 .IP \[bu] 2 75819 Remove revoked credentials, allow oauth proxy config and update docs 75820 .RE 75821 .IP \[bu] 2 75822 B2 75823 .RS 2 75824 .IP \[bu] 2 75825 Reduce minimum chunk size to 5MB 75826 .RE 75827 .IP \[bu] 2 75828 Drive 75829 .RS 2 75830 .IP \[bu] 2 75831 Add team drive support 75832 .IP \[bu] 2 75833 Reduce bandwidth by adding fields for partial responses - Martin 75834 Kristensen 75835 .IP \[bu] 2 75836 Implement --drive-shared-with-me flag to view shared with me files - 75837 Danny Tsai 75838 .IP \[bu] 2 75839 Add --drive-trashed-only to read only the files in the trash 75840 .IP \[bu] 2 75841 Remove obsolete --drive-full-list 75842 .IP \[bu] 2 75843 Add missing seek to start on retries of chunked uploads 75844 .IP \[bu] 2 75845 Fix stats accounting for upload 75846 .IP \[bu] 2 75847 Convert / in names to a unicode equivalent (\[uFF0F]) 75848 .IP \[bu] 2 75849 Poll for Google Drive changes when mounted 75850 .RE 75851 .IP \[bu] 2 75852 OneDrive 75853 .RS 2 75854 .IP \[bu] 2 75855 Fix the uploading of files with spaces 75856 .IP \[bu] 2 75857 Fix initialization order for token renewer 75858 .IP \[bu] 2 75859 Display speeds accurately when uploading - Yoni Jah 75860 .IP \[bu] 2 75861 Swap to using http://localhost:53682/ as redirect URL - Michael Ledin 75862 .IP \[bu] 2 75863 Retry on token expired error, reset upload body on retry - Yoni Jah 75864 .RE 75865 .IP \[bu] 2 75866 Google Cloud Storage 75867 .RS 2 75868 .IP \[bu] 2 75869 Add ability to specify location and storage class via config and command 75870 line - thanks gdm85 75871 .IP \[bu] 2 75872 Create container if necessary on server-side copy 75873 .IP \[bu] 2 75874 Increase directory listing chunk to 1000 to increase performance 75875 .IP \[bu] 2 75876 Obtain a refresh token for GCS - Steven Lu 75877 .RE 75878 .IP \[bu] 2 75879 Yandex 75880 .RS 2 75881 .IP \[bu] 2 75882 Fix the name reported in log messages (was empty) 75883 .IP \[bu] 2 75884 Correct error return for listing empty directory 75885 .RE 75886 .IP \[bu] 2 75887 Dropbox 75888 .RS 2 75889 .IP \[bu] 2 75890 Rewritten to use the v2 API 75891 .RS 2 75892 .IP \[bu] 2 75893 Now supports ModTime 75894 .RS 2 75895 .IP \[bu] 2 75896 Can only set by uploading the file again 75897 .IP \[bu] 2 75898 If you uploaded with an old rclone, rclone may upload everything again 75899 .IP \[bu] 2 75900 Use \f[C]--size-only\f[R] or \f[C]--checksum\f[R] to avoid this 75901 .RE 75902 .IP \[bu] 2 75903 Now supports the Dropbox content hashing scheme 75904 .IP \[bu] 2 75905 Now supports low level retries 75906 .RE 75907 .RE 75908 .IP \[bu] 2 75909 S3 75910 .RS 2 75911 .IP \[bu] 2 75912 Work around eventual consistency in bucket creation 75913 .IP \[bu] 2 75914 Create container if necessary on server-side copy 75915 .IP \[bu] 2 75916 Add us-east-2 (Ohio) and eu-west-2 (London) S3 regions - Zahiar Ahmed 75917 .RE 75918 .IP \[bu] 2 75919 Swift, Hubic 75920 .RS 2 75921 .IP \[bu] 2 75922 Fix zero length directory markers showing in the subdirectory listing 75923 .RS 2 75924 .IP \[bu] 2 75925 this caused lots of duplicate transfers 75926 .RE 75927 .IP \[bu] 2 75928 Fix paged directory listings 75929 .RS 2 75930 .IP \[bu] 2 75931 this caused duplicate directory errors 75932 .RE 75933 .IP \[bu] 2 75934 Create container if necessary on server-side copy 75935 .IP \[bu] 2 75936 Increase directory listing chunk to 1000 to increase performance 75937 .IP \[bu] 2 75938 Make sensible error if the user forgets the container 75939 .RE 75940 .IP \[bu] 2 75941 SFTP 75942 .RS 2 75943 .IP \[bu] 2 75944 Add support for using ssh key files 75945 .IP \[bu] 2 75946 Fix under Windows 75947 .IP \[bu] 2 75948 Fix ssh agent on Windows 75949 .IP \[bu] 2 75950 Adapt to latest version of library - Igor Kharin 75951 .RE 75952 .SS v1.36 - 2017-03-18 75953 .IP \[bu] 2 75954 New Features 75955 .RS 2 75956 .IP \[bu] 2 75957 SFTP remote (Jack Schmidt) 75958 .IP \[bu] 2 75959 Re-implement sync routine to work a directory at a time reducing memory 75960 usage 75961 .IP \[bu] 2 75962 Logging revamped to be more inline with rsync - now much quieter * -v 75963 only shows transfers * -vv is for full debug * --syslog to log to syslog 75964 on capable platforms 75965 .IP \[bu] 2 75966 Implement --backup-dir and --suffix 75967 .IP \[bu] 2 75968 Implement --track-renames (initial implementation by Bj\[/o]rn Erik 75969 Pedersen) 75970 .IP \[bu] 2 75971 Add time-based bandwidth limits (Lukas Loesche) 75972 .IP \[bu] 2 75973 rclone cryptcheck: checks integrity of crypt remotes 75974 .IP \[bu] 2 75975 Allow all config file variables and options to be set from environment 75976 variables 75977 .IP \[bu] 2 75978 Add --buffer-size parameter to control buffer size for copy 75979 .IP \[bu] 2 75980 Make --delete-after the default 75981 .IP \[bu] 2 75982 Add --ignore-checksum flag (fixed by Hisham Zarka) 75983 .IP \[bu] 2 75984 rclone check: Add --download flag to check all the data, not just hashes 75985 .IP \[bu] 2 75986 rclone cat: add --head, --tail, --offset, --count and --discard 75987 .IP \[bu] 2 75988 rclone config: when choosing from a list, allow the value to be entered 75989 too 75990 .IP \[bu] 2 75991 rclone config: allow rename and copy of remotes 75992 .IP \[bu] 2 75993 rclone obscure: for generating encrypted passwords for rclone\[aq]s 75994 config (T.C. 75995 Ferguson) 75996 .IP \[bu] 2 75997 Comply with XDG Base Directory specification (Dario Giovannetti) 75998 .RS 2 75999 .IP \[bu] 2 76000 this moves the default location of the config file in a backwards 76001 compatible way 76002 .RE 76003 .IP \[bu] 2 76004 Release changes 76005 .RS 2 76006 .IP \[bu] 2 76007 Ubuntu snap support (Dedsec1) 76008 .IP \[bu] 2 76009 Compile with go 1.8 76010 .IP \[bu] 2 76011 MIPS/Linux big and little endian support 76012 .RE 76013 .RE 76014 .IP \[bu] 2 76015 Bug Fixes 76016 .RS 2 76017 .IP \[bu] 2 76018 Fix copyto copying things to the wrong place if the destination dir 76019 didn\[aq]t exist 76020 .IP \[bu] 2 76021 Fix parsing of remotes in moveto and copyto 76022 .IP \[bu] 2 76023 Fix --delete-before deleting files on copy 76024 .IP \[bu] 2 76025 Fix --files-from with an empty file copying everything 76026 .IP \[bu] 2 76027 Fix sync: don\[aq]t update mod times if --dry-run set 76028 .IP \[bu] 2 76029 Fix MimeType propagation 76030 .IP \[bu] 2 76031 Fix filters to add ** rules to directory rules 76032 .RE 76033 .IP \[bu] 2 76034 Local 76035 .RS 2 76036 .IP \[bu] 2 76037 Implement -L, --copy-links flag to allow rclone to follow symlinks 76038 .IP \[bu] 2 76039 Open files in write only mode so rclone can write to an rclone mount 76040 .IP \[bu] 2 76041 Fix unnormalised unicode causing problems reading directories 76042 .IP \[bu] 2 76043 Fix interaction between -x flag and --max-depth 76044 .RE 76045 .IP \[bu] 2 76046 Mount 76047 .RS 2 76048 .IP \[bu] 2 76049 Implement proper directory handling (mkdir, rmdir, renaming) 76050 .IP \[bu] 2 76051 Make include and exclude filters apply to mount 76052 .IP \[bu] 2 76053 Implement read and write async buffers - control with --buffer-size 76054 .IP \[bu] 2 76055 Fix fsync on for directories 76056 .IP \[bu] 2 76057 Fix retry on network failure when reading off crypt 76058 .RE 76059 .IP \[bu] 2 76060 Crypt 76061 .RS 2 76062 .IP \[bu] 2 76063 Add --crypt-show-mapping to show encrypted file mapping 76064 .IP \[bu] 2 76065 Fix crypt writer getting stuck in a loop 76066 .RS 2 76067 .IP \[bu] 2 76068 \f[B]IMPORTANT\f[R] this bug had the potential to cause data corruption 76069 when 76070 .RS 2 76071 .IP \[bu] 2 76072 reading data from a network based remote and 76073 .IP \[bu] 2 76074 writing to a crypt on Google Drive 76075 .RE 76076 .IP \[bu] 2 76077 Use the cryptcheck command to validate your data if you are concerned 76078 .IP \[bu] 2 76079 If syncing two crypt remotes, sync the unencrypted remote 76080 .RE 76081 .RE 76082 .IP \[bu] 2 76083 Amazon Drive 76084 .RS 2 76085 .IP \[bu] 2 76086 Fix panics on Move (rename) 76087 .IP \[bu] 2 76088 Fix panic on token expiry 76089 .RE 76090 .IP \[bu] 2 76091 B2 76092 .RS 2 76093 .IP \[bu] 2 76094 Fix inconsistent listings and rclone check 76095 .IP \[bu] 2 76096 Fix uploading empty files with go1.8 76097 .IP \[bu] 2 76098 Constrain memory usage when doing multipart uploads 76099 .IP \[bu] 2 76100 Fix upload url not being refreshed properly 76101 .RE 76102 .IP \[bu] 2 76103 Drive 76104 .RS 2 76105 .IP \[bu] 2 76106 Fix Rmdir on directories with trashed files 76107 .IP \[bu] 2 76108 Fix \[dq]Ignoring unknown object\[dq] when downloading 76109 .IP \[bu] 2 76110 Add --drive-list-chunk 76111 .IP \[bu] 2 76112 Add --drive-skip-gdocs (K\['a]roly Ol\['a]h) 76113 .RE 76114 .IP \[bu] 2 76115 OneDrive 76116 .RS 2 76117 .IP \[bu] 2 76118 Implement Move 76119 .IP \[bu] 2 76120 Fix Copy 76121 .RS 2 76122 .IP \[bu] 2 76123 Fix overwrite detection in Copy 76124 .IP \[bu] 2 76125 Fix waitForJob to parse errors correctly 76126 .RE 76127 .IP \[bu] 2 76128 Use token renewer to stop auth errors on long uploads 76129 .IP \[bu] 2 76130 Fix uploading empty files with go1.8 76131 .RE 76132 .IP \[bu] 2 76133 Google Cloud Storage 76134 .RS 2 76135 .IP \[bu] 2 76136 Fix depth 1 directory listings 76137 .RE 76138 .IP \[bu] 2 76139 Yandex 76140 .RS 2 76141 .IP \[bu] 2 76142 Fix single level directory listing 76143 .RE 76144 .IP \[bu] 2 76145 Dropbox 76146 .RS 2 76147 .IP \[bu] 2 76148 Normalise the case for single level directory listings 76149 .IP \[bu] 2 76150 Fix depth 1 listing 76151 .RE 76152 .IP \[bu] 2 76153 S3 76154 .RS 2 76155 .IP \[bu] 2 76156 Added ca-central-1 region (Jon Yergatian) 76157 .RE 76158 .SS v1.35 - 2017-01-02 76159 .IP \[bu] 2 76160 New Features 76161 .RS 2 76162 .IP \[bu] 2 76163 moveto and copyto commands for choosing a destination name on copy/move 76164 .IP \[bu] 2 76165 rmdirs command to recursively delete empty directories 76166 .IP \[bu] 2 76167 Allow repeated --include/--exclude/--filter options 76168 .IP \[bu] 2 76169 Only show transfer stats on commands which transfer stuff 76170 .RS 2 76171 .IP \[bu] 2 76172 show stats on any command using the \f[C]--stats\f[R] flag 76173 .RE 76174 .IP \[bu] 2 76175 Allow overlapping directories in move when server-side dir move is 76176 supported 76177 .IP \[bu] 2 76178 Add --stats-unit option - thanks Scott McGillivray 76179 .RE 76180 .IP \[bu] 2 76181 Bug Fixes 76182 .RS 2 76183 .IP \[bu] 2 76184 Fix the config file being overwritten when two rclone instances are 76185 running 76186 .IP \[bu] 2 76187 Make rclone lsd obey the filters properly 76188 .IP \[bu] 2 76189 Fix compilation on mips 76190 .IP \[bu] 2 76191 Fix not transferring files that don\[aq]t differ in size 76192 .IP \[bu] 2 76193 Fix panic on nil retry/fatal error 76194 .RE 76195 .IP \[bu] 2 76196 Mount 76197 .RS 2 76198 .IP \[bu] 2 76199 Retry reads on error - should help with reliability a lot 76200 .IP \[bu] 2 76201 Report the modification times for directories from the remote 76202 .IP \[bu] 2 76203 Add bandwidth accounting and limiting (fixes --bwlimit) 76204 .IP \[bu] 2 76205 If --stats provided will show stats and which files are transferring 76206 .IP \[bu] 2 76207 Support R/W files if truncate is set. 76208 .IP \[bu] 2 76209 Implement statfs interface so df works 76210 .IP \[bu] 2 76211 Note that write is now supported on Amazon Drive 76212 .IP \[bu] 2 76213 Report number of blocks in a file - thanks Stefan Breunig 76214 .RE 76215 .IP \[bu] 2 76216 Crypt 76217 .RS 2 76218 .IP \[bu] 2 76219 Prevent the user pointing crypt at itself 76220 .IP \[bu] 2 76221 Fix failed to authenticate decrypted block errors 76222 .RS 2 76223 .IP \[bu] 2 76224 these will now return the underlying unexpected EOF instead 76225 .RE 76226 .RE 76227 .IP \[bu] 2 76228 Amazon Drive 76229 .RS 2 76230 .IP \[bu] 2 76231 Add support for server-side move and directory move - thanks Stefan 76232 Breunig 76233 .IP \[bu] 2 76234 Fix nil pointer deref on size attribute 76235 .RE 76236 .IP \[bu] 2 76237 B2 76238 .RS 2 76239 .IP \[bu] 2 76240 Use new prefix and delimiter parameters in directory listings 76241 .RS 2 76242 .IP \[bu] 2 76243 This makes --max-depth 1 dir listings as used in mount much faster 76244 .RE 76245 .IP \[bu] 2 76246 Reauth the account while doing uploads too - should help with token 76247 expiry 76248 .RE 76249 .IP \[bu] 2 76250 Drive 76251 .RS 2 76252 .IP \[bu] 2 76253 Make DirMove more efficient and complain about moving the root 76254 .IP \[bu] 2 76255 Create destination directory on Move() 76256 .RE 76257 .SS v1.34 - 2016-11-06 76258 .IP \[bu] 2 76259 New Features 76260 .RS 2 76261 .IP \[bu] 2 76262 Stop single file and \f[C]--files-from\f[R] operations iterating through 76263 the source bucket. 76264 .IP \[bu] 2 76265 Stop removing failed upload to cloud storage remotes 76266 .IP \[bu] 2 76267 Make ContentType be preserved for cloud to cloud copies 76268 .IP \[bu] 2 76269 Add support to toggle bandwidth limits via SIGUSR2 - thanks Marco 76270 Paganini 76271 .IP \[bu] 2 76272 \f[C]rclone check\f[R] shows count of hashes that couldn\[aq]t be 76273 checked 76274 .IP \[bu] 2 76275 \f[C]rclone listremotes\f[R] command 76276 .IP \[bu] 2 76277 Support linux/arm64 build - thanks Fredrik Fornwall 76278 .IP \[bu] 2 76279 Remove \f[C]Authorization:\f[R] lines from \f[C]--dump-headers\f[R] 76280 output 76281 .RE 76282 .IP \[bu] 2 76283 Bug Fixes 76284 .RS 2 76285 .IP \[bu] 2 76286 Ignore files with control characters in the names 76287 .IP \[bu] 2 76288 Fix \f[C]rclone move\f[R] command 76289 .RS 2 76290 .IP \[bu] 2 76291 Delete src files which already existed in dst 76292 .IP \[bu] 2 76293 Fix deletion of src file when dst file older 76294 .RE 76295 .IP \[bu] 2 76296 Fix \f[C]rclone check\f[R] on encrypted file systems 76297 .IP \[bu] 2 76298 Make failed uploads not count as \[dq]Transferred\[dq] 76299 .IP \[bu] 2 76300 Make sure high level retries show with \f[C]-q\f[R] 76301 .IP \[bu] 2 76302 Use a vendor directory with godep for repeatable builds 76303 .RE 76304 .IP \[bu] 2 76305 \f[C]rclone mount\f[R] - FUSE 76306 .RS 2 76307 .IP \[bu] 2 76308 Implement FUSE mount options 76309 .RS 2 76310 .IP \[bu] 2 76311 \f[C]--no-modtime\f[R], \f[C]--debug-fuse\f[R], \f[C]--read-only\f[R], 76312 \f[C]--allow-non-empty\f[R], \f[C]--allow-root\f[R], 76313 \f[C]--allow-other\f[R] 76314 .IP \[bu] 2 76315 \f[C]--default-permissions\f[R], \f[C]--write-back-cache\f[R], 76316 \f[C]--max-read-ahead\f[R], \f[C]--umask\f[R], \f[C]--uid\f[R], 76317 \f[C]--gid\f[R] 76318 .RE 76319 .IP \[bu] 2 76320 Add \f[C]--dir-cache-time\f[R] to control caching of directory entries 76321 .IP \[bu] 2 76322 Implement seek for files opened for read (useful for video players) 76323 .RS 2 76324 .IP \[bu] 2 76325 with \f[C]-no-seek\f[R] flag to disable 76326 .RE 76327 .IP \[bu] 2 76328 Fix crash on 32 bit ARM (alignment of 64 bit counter) 76329 .IP \[bu] 2 76330 \&...and many more internal fixes and improvements! 76331 .RE 76332 .IP \[bu] 2 76333 Crypt 76334 .RS 2 76335 .IP \[bu] 2 76336 Don\[aq]t show encrypted password in configurator to stop confusion 76337 .RE 76338 .IP \[bu] 2 76339 Amazon Drive 76340 .RS 2 76341 .IP \[bu] 2 76342 New wait for upload option \f[C]--acd-upload-wait-per-gb\f[R] 76343 .RS 2 76344 .IP \[bu] 2 76345 upload timeouts scale by file size and can be disabled 76346 .RE 76347 .IP \[bu] 2 76348 Add 502 Bad Gateway to list of errors we retry 76349 .IP \[bu] 2 76350 Fix overwriting a file with a zero length file 76351 .IP \[bu] 2 76352 Fix ACD file size warning limit - thanks Felix B\[:u]nemann 76353 .RE 76354 .IP \[bu] 2 76355 Local 76356 .RS 2 76357 .IP \[bu] 2 76358 Unix: implement \f[C]-x\f[R]/\f[C]--one-file-system\f[R] to stay on a 76359 single file system 76360 .RS 2 76361 .IP \[bu] 2 76362 thanks Durval Menezes and Luiz Carlos Rumbelsperger Viana 76363 .RE 76364 .IP \[bu] 2 76365 Windows: ignore the symlink bit on files 76366 .IP \[bu] 2 76367 Windows: Ignore directory-based junction points 76368 .RE 76369 .IP \[bu] 2 76370 B2 76371 .RS 2 76372 .IP \[bu] 2 76373 Make sure each upload has at least one upload slot - fixes strange 76374 upload stats 76375 .IP \[bu] 2 76376 Fix uploads when using crypt 76377 .IP \[bu] 2 76378 Fix download of large files (sha1 mismatch) 76379 .IP \[bu] 2 76380 Return error when we try to create a bucket which someone else owns 76381 .IP \[bu] 2 76382 Update B2 docs with Data usage, and Crypt section - thanks Tomasz Mazur 76383 .RE 76384 .IP \[bu] 2 76385 S3 76386 .RS 2 76387 .IP \[bu] 2 76388 Command line and config file support for 76389 .RS 2 76390 .IP \[bu] 2 76391 Setting/overriding ACL - thanks Radek \[vS]enfeld 76392 .IP \[bu] 2 76393 Setting storage class - thanks Asko Tamm 76394 .RE 76395 .RE 76396 .IP \[bu] 2 76397 Drive 76398 .RS 2 76399 .IP \[bu] 2 76400 Make exponential backoff work exactly as per Google specification 76401 .IP \[bu] 2 76402 add \f[C].epub\f[R], \f[C].odp\f[R] and \f[C].tsv\f[R] as export 76403 formats. 76404 .RE 76405 .IP \[bu] 2 76406 Swift 76407 .RS 2 76408 .IP \[bu] 2 76409 Don\[aq]t read metadata for directory marker objects 76410 .RE 76411 .SS v1.33 - 2016-08-24 76412 .IP \[bu] 2 76413 New Features 76414 .RS 2 76415 .IP \[bu] 2 76416 Implement encryption 76417 .RS 2 76418 .IP \[bu] 2 76419 data encrypted in NACL secretbox format 76420 .IP \[bu] 2 76421 with optional file name encryption 76422 .RE 76423 .IP \[bu] 2 76424 New commands 76425 .RS 2 76426 .IP \[bu] 2 76427 rclone mount - implements FUSE mounting of remotes (EXPERIMENTAL) 76428 .RS 2 76429 .IP \[bu] 2 76430 works on Linux, FreeBSD and OS X (need testers for the last 2!) 76431 .RE 76432 .IP \[bu] 2 76433 rclone cat - outputs remote file or files to the terminal 76434 .IP \[bu] 2 76435 rclone genautocomplete - command to make a bash completion script for 76436 rclone 76437 .RE 76438 .IP \[bu] 2 76439 Editing a remote using \f[C]rclone config\f[R] now goes through the 76440 wizard 76441 .IP \[bu] 2 76442 Compile with go 1.7 - this fixes rclone on macOS Sierra and on 386 76443 processors 76444 .IP \[bu] 2 76445 Use cobra for sub commands and docs generation 76446 .RE 76447 .IP \[bu] 2 76448 drive 76449 .RS 2 76450 .IP \[bu] 2 76451 Document how to make your own client_id 76452 .RE 76453 .IP \[bu] 2 76454 s3 76455 .RS 2 76456 .IP \[bu] 2 76457 User-configurable Amazon S3 ACL (thanks Radek \[vS]enfeld) 76458 .RE 76459 .IP \[bu] 2 76460 b2 76461 .RS 2 76462 .IP \[bu] 2 76463 Fix stats accounting for upload - no more jumping to 100% done 76464 .IP \[bu] 2 76465 On cleanup delete hide marker if it is the current file 76466 .IP \[bu] 2 76467 New B2 API endpoint (thanks Per Cederberg) 76468 .IP \[bu] 2 76469 Set maximum backoff to 5 Minutes 76470 .RE 76471 .IP \[bu] 2 76472 onedrive 76473 .RS 2 76474 .IP \[bu] 2 76475 Fix URL escaping in file names - e.g. 76476 uploading files with \f[C]+\f[R] in them. 76477 .RE 76478 .IP \[bu] 2 76479 amazon cloud drive 76480 .RS 2 76481 .IP \[bu] 2 76482 Fix token expiry during large uploads 76483 .IP \[bu] 2 76484 Work around 408 REQUEST_TIMEOUT and 504 GATEWAY_TIMEOUT errors 76485 .RE 76486 .IP \[bu] 2 76487 local 76488 .RS 2 76489 .IP \[bu] 2 76490 Fix filenames with invalid UTF-8 not being uploaded 76491 .IP \[bu] 2 76492 Fix problem with some UTF-8 characters on OS X 76493 .RE 76494 .SS v1.32 - 2016-07-13 76495 .IP \[bu] 2 76496 Backblaze B2 76497 .RS 2 76498 .IP \[bu] 2 76499 Fix upload of files large files not in root 76500 .RE 76501 .SS v1.31 - 2016-07-13 76502 .IP \[bu] 2 76503 New Features 76504 .RS 2 76505 .IP \[bu] 2 76506 Reduce memory on sync by about 50% 76507 .IP \[bu] 2 76508 Implement --no-traverse flag to stop copy traversing the destination 76509 remote. 76510 .RS 2 76511 .IP \[bu] 2 76512 This can be used to reduce memory usage down to the smallest possible. 76513 .IP \[bu] 2 76514 Useful to copy a small number of files into a large destination folder. 76515 .RE 76516 .IP \[bu] 2 76517 Implement cleanup command for emptying trash / removing old versions of 76518 files 76519 .RS 2 76520 .IP \[bu] 2 76521 Currently B2 only 76522 .RE 76523 .IP \[bu] 2 76524 Single file handling improved 76525 .RS 2 76526 .IP \[bu] 2 76527 Now copied with --files-from 76528 .IP \[bu] 2 76529 Automatically sets --no-traverse when copying a single file 76530 .RE 76531 .IP \[bu] 2 76532 Info on using installing with ansible - thanks Stefan Weichinger 76533 .IP \[bu] 2 76534 Implement --no-update-modtime flag to stop rclone fixing the remote 76535 modified times. 76536 .RE 76537 .IP \[bu] 2 76538 Bug Fixes 76539 .RS 2 76540 .IP \[bu] 2 76541 Fix move command - stop it running for overlapping Fses - this was 76542 causing data loss. 76543 .RE 76544 .IP \[bu] 2 76545 Local 76546 .RS 2 76547 .IP \[bu] 2 76548 Fix incomplete hashes - this was causing problems for B2. 76549 .RE 76550 .IP \[bu] 2 76551 Amazon Drive 76552 .RS 2 76553 .IP \[bu] 2 76554 Rename Amazon Cloud Drive to Amazon Drive - no changes to config file 76555 needed. 76556 .RE 76557 .IP \[bu] 2 76558 Swift 76559 .RS 2 76560 .IP \[bu] 2 76561 Add support for non-default project domain - thanks Antonio Messina. 76562 .RE 76563 .IP \[bu] 2 76564 S3 76565 .RS 2 76566 .IP \[bu] 2 76567 Add instructions on how to use rclone with minio. 76568 .IP \[bu] 2 76569 Add ap-northeast-2 (Seoul) and ap-south-1 (Mumbai) regions. 76570 .IP \[bu] 2 76571 Skip setting the modified time for objects > 5GB as it isn\[aq]t 76572 possible. 76573 .RE 76574 .IP \[bu] 2 76575 Backblaze B2 76576 .RS 2 76577 .IP \[bu] 2 76578 Add --b2-versions flag so old versions can be listed and retrieved. 76579 .IP \[bu] 2 76580 Treat 403 errors (e.g. 76581 cap exceeded) as fatal. 76582 .IP \[bu] 2 76583 Implement cleanup command for deleting old file versions. 76584 .IP \[bu] 2 76585 Make error handling compliant with B2 integrations notes. 76586 .IP \[bu] 2 76587 Fix handling of token expiry. 76588 .IP \[bu] 2 76589 Implement --b2-test-mode to set \f[C]X-Bz-Test-Mode\f[R] header. 76590 .IP \[bu] 2 76591 Set cutoff for chunked upload to 200MB as per B2 guidelines. 76592 .IP \[bu] 2 76593 Make upload multi-threaded. 76594 .RE 76595 .IP \[bu] 2 76596 Dropbox 76597 .RS 2 76598 .IP \[bu] 2 76599 Don\[aq]t retry 461 errors. 76600 .RE 76601 .SS v1.30 - 2016-06-18 76602 .IP \[bu] 2 76603 New Features 76604 .RS 2 76605 .IP \[bu] 2 76606 Directory listing code reworked for more features and better error 76607 reporting (thanks to Klaus Post for help). 76608 This enables 76609 .RS 2 76610 .IP \[bu] 2 76611 Directory include filtering for efficiency 76612 .IP \[bu] 2 76613 --max-depth parameter 76614 .IP \[bu] 2 76615 Better error reporting 76616 .IP \[bu] 2 76617 More to come 76618 .RE 76619 .IP \[bu] 2 76620 Retry more errors 76621 .IP \[bu] 2 76622 Add --ignore-size flag - for uploading images to onedrive 76623 .IP \[bu] 2 76624 Log -v output to stdout by default 76625 .IP \[bu] 2 76626 Display the transfer stats in more human-readable form 76627 .IP \[bu] 2 76628 Make 0 size files specifiable with \f[C]--max-size 0b\f[R] 76629 .IP \[bu] 2 76630 Add \f[C]b\f[R] suffix so we can specify bytes in --bwlimit, --min-size, 76631 etc. 76632 .IP \[bu] 2 76633 Use \[dq]password:\[dq] instead of \[dq]password>\[dq] prompt - thanks 76634 Klaus Post and Leigh Klotz 76635 .RE 76636 .IP \[bu] 2 76637 Bug Fixes 76638 .RS 2 76639 .IP \[bu] 2 76640 Fix retry doing one too many retries 76641 .RE 76642 .IP \[bu] 2 76643 Local 76644 .RS 2 76645 .IP \[bu] 2 76646 Fix problems with OS X and UTF-8 characters 76647 .RE 76648 .IP \[bu] 2 76649 Amazon Drive 76650 .RS 2 76651 .IP \[bu] 2 76652 Check a file exists before uploading to help with 408 Conflict errors 76653 .IP \[bu] 2 76654 Reauth on 401 errors - this has been causing a lot of problems 76655 .IP \[bu] 2 76656 Work around spurious 403 errors 76657 .IP \[bu] 2 76658 Restart directory listings on error 76659 .RE 76660 .IP \[bu] 2 76661 Google Drive 76662 .RS 2 76663 .IP \[bu] 2 76664 Check a file exists before uploading to help with duplicates 76665 .IP \[bu] 2 76666 Fix retry of multipart uploads 76667 .RE 76668 .IP \[bu] 2 76669 Backblaze B2 76670 .RS 2 76671 .IP \[bu] 2 76672 Implement large file uploading 76673 .RE 76674 .IP \[bu] 2 76675 S3 76676 .RS 2 76677 .IP \[bu] 2 76678 Add AES256 server-side encryption for - thanks Justin R. 76679 Wilson 76680 .RE 76681 .IP \[bu] 2 76682 Google Cloud Storage 76683 .RS 2 76684 .IP \[bu] 2 76685 Make sure we don\[aq]t use conflicting content types on upload 76686 .IP \[bu] 2 76687 Add service account support - thanks Michal Witkowski 76688 .RE 76689 .IP \[bu] 2 76690 Swift 76691 .RS 2 76692 .IP \[bu] 2 76693 Add auth version parameter 76694 .IP \[bu] 2 76695 Add domain option for openstack (v3 auth) - thanks Fabian Ruff 76696 .RE 76697 .SS v1.29 - 2016-04-18 76698 .IP \[bu] 2 76699 New Features 76700 .RS 2 76701 .IP \[bu] 2 76702 Implement \f[C]-I, --ignore-times\f[R] for unconditional upload 76703 .IP \[bu] 2 76704 Improve \f[C]dedupe\f[R]command 76705 .RS 2 76706 .IP \[bu] 2 76707 Now removes identical copies without asking 76708 .IP \[bu] 2 76709 Now obeys \f[C]--dry-run\f[R] 76710 .IP \[bu] 2 76711 Implement \f[C]--dedupe-mode\f[R] for non interactive running 76712 .RS 2 76713 .IP \[bu] 2 76714 \f[C]--dedupe-mode interactive\f[R] - interactive the default. 76715 .IP \[bu] 2 76716 \f[C]--dedupe-mode skip\f[R] - removes identical files then skips 76717 anything left. 76718 .IP \[bu] 2 76719 \f[C]--dedupe-mode first\f[R] - removes identical files then keeps the 76720 first one. 76721 .IP \[bu] 2 76722 \f[C]--dedupe-mode newest\f[R] - removes identical files then keeps the 76723 newest one. 76724 .IP \[bu] 2 76725 \f[C]--dedupe-mode oldest\f[R] - removes identical files then keeps the 76726 oldest one. 76727 .IP \[bu] 2 76728 \f[C]--dedupe-mode rename\f[R] - removes identical files then renames 76729 the rest to be different. 76730 .RE 76731 .RE 76732 .RE 76733 .IP \[bu] 2 76734 Bug fixes 76735 .RS 2 76736 .IP \[bu] 2 76737 Make rclone check obey the \f[C]--size-only\f[R] flag. 76738 .IP \[bu] 2 76739 Use \[dq]application/octet-stream\[dq] if discovered mime type is 76740 invalid. 76741 .IP \[bu] 2 76742 Fix missing \[dq]quit\[dq] option when there are no remotes. 76743 .RE 76744 .IP \[bu] 2 76745 Google Drive 76746 .RS 2 76747 .IP \[bu] 2 76748 Increase default chunk size to 8 MB - increases upload speed of big 76749 files 76750 .IP \[bu] 2 76751 Speed up directory listings and make more reliable 76752 .IP \[bu] 2 76753 Add missing retries for Move and DirMove - increases reliability 76754 .IP \[bu] 2 76755 Preserve mime type on file update 76756 .RE 76757 .IP \[bu] 2 76758 Backblaze B2 76759 .RS 2 76760 .IP \[bu] 2 76761 Enable mod time syncing 76762 .RS 2 76763 .IP \[bu] 2 76764 This means that B2 will now check modification times 76765 .IP \[bu] 2 76766 It will upload new files to update the modification times 76767 .IP \[bu] 2 76768 (there isn\[aq]t an API to just set the mod time.) 76769 .IP \[bu] 2 76770 If you want the old behaviour use \f[C]--size-only\f[R]. 76771 .RE 76772 .IP \[bu] 2 76773 Update API to new version 76774 .IP \[bu] 2 76775 Fix parsing of mod time when not in metadata 76776 .RE 76777 .IP \[bu] 2 76778 Swift/Hubic 76779 .RS 2 76780 .IP \[bu] 2 76781 Don\[aq]t return an MD5SUM for static large objects 76782 .RE 76783 .IP \[bu] 2 76784 S3 76785 .RS 2 76786 .IP \[bu] 2 76787 Fix uploading files bigger than 50GB 76788 .RE 76789 .SS v1.28 - 2016-03-01 76790 .IP \[bu] 2 76791 New Features 76792 .RS 2 76793 .IP \[bu] 2 76794 Configuration file encryption - thanks Klaus Post 76795 .IP \[bu] 2 76796 Improve \f[C]rclone config\f[R] adding more help and making it easier to 76797 understand 76798 .IP \[bu] 2 76799 Implement \f[C]-u\f[R]/\f[C]--update\f[R] so creation times can be used 76800 on all remotes 76801 .IP \[bu] 2 76802 Implement \f[C]--low-level-retries\f[R] flag 76803 .IP \[bu] 2 76804 Optionally disable gzip compression on downloads with 76805 \f[C]--no-gzip-encoding\f[R] 76806 .RE 76807 .IP \[bu] 2 76808 Bug fixes 76809 .RS 2 76810 .IP \[bu] 2 76811 Don\[aq]t make directories if \f[C]--dry-run\f[R] set 76812 .IP \[bu] 2 76813 Fix and document the \f[C]move\f[R] command 76814 .IP \[bu] 2 76815 Fix redirecting stderr on unix-like OSes when using \f[C]--log-file\f[R] 76816 .IP \[bu] 2 76817 Fix \f[C]delete\f[R] command to wait until all finished - fixes missing 76818 deletes. 76819 .RE 76820 .IP \[bu] 2 76821 Backblaze B2 76822 .RS 2 76823 .IP \[bu] 2 76824 Use one upload URL per go routine fixes 76825 \f[C]more than one upload using auth token\f[R] 76826 .IP \[bu] 2 76827 Add pacing, retries and reauthentication - fixes token expiry problems 76828 .IP \[bu] 2 76829 Upload without using a temporary file from local (and remotes which 76830 support SHA1) 76831 .IP \[bu] 2 76832 Fix reading metadata for all files when it shouldn\[aq]t have been 76833 .RE 76834 .IP \[bu] 2 76835 Drive 76836 .RS 2 76837 .IP \[bu] 2 76838 Fix listing drive documents at root 76839 .IP \[bu] 2 76840 Disable copy and move for Google docs 76841 .RE 76842 .IP \[bu] 2 76843 Swift 76844 .RS 2 76845 .IP \[bu] 2 76846 Fix uploading of chunked files with non ASCII characters 76847 .IP \[bu] 2 76848 Allow setting of \f[C]storage_url\f[R] in the config - thanks Xavier 76849 Lucas 76850 .RE 76851 .IP \[bu] 2 76852 S3 76853 .RS 2 76854 .IP \[bu] 2 76855 Allow IAM role and credentials from environment variables - thanks Brian 76856 Stengaard 76857 .IP \[bu] 2 76858 Allow low privilege users to use S3 (check if directory exists during 76859 Mkdir) - thanks Jakub Gedeon 76860 .RE 76861 .IP \[bu] 2 76862 Amazon Drive 76863 .RS 2 76864 .IP \[bu] 2 76865 Retry on more things to make directory listings more reliable 76866 .RE 76867 .SS v1.27 - 2016-01-31 76868 .IP \[bu] 2 76869 New Features 76870 .RS 2 76871 .IP \[bu] 2 76872 Easier headless configuration with \f[C]rclone authorize\f[R] 76873 .IP \[bu] 2 76874 Add support for multiple hash types - we now check SHA1 as well as MD5 76875 hashes. 76876 .IP \[bu] 2 76877 \f[C]delete\f[R] command which does obey the filters (unlike 76878 \f[C]purge\f[R]) 76879 .IP \[bu] 2 76880 \f[C]dedupe\f[R] command to deduplicate a remote. 76881 Useful with Google Drive. 76882 .IP \[bu] 2 76883 Add \f[C]--ignore-existing\f[R] flag to skip all files that exist on 76884 destination. 76885 .IP \[bu] 2 76886 Add \f[C]--delete-before\f[R], \f[C]--delete-during\f[R], 76887 \f[C]--delete-after\f[R] flags. 76888 .IP \[bu] 2 76889 Add \f[C]--memprofile\f[R] flag to debug memory use. 76890 .IP \[bu] 2 76891 Warn the user about files with same name but different case 76892 .IP \[bu] 2 76893 Make \f[C]--include\f[R] rules add their implicit exclude * at the end 76894 of the filter list 76895 .IP \[bu] 2 76896 Deprecate compiling with go1.3 76897 .RE 76898 .IP \[bu] 2 76899 Amazon Drive 76900 .RS 2 76901 .IP \[bu] 2 76902 Fix download of files > 10 GB 76903 .IP \[bu] 2 76904 Fix directory traversal (\[dq]Next token is expired\[dq]) for large 76905 directory listings 76906 .IP \[bu] 2 76907 Remove 409 conflict from error codes we will retry - stops very long 76908 pauses 76909 .RE 76910 .IP \[bu] 2 76911 Backblaze B2 76912 .RS 2 76913 .IP \[bu] 2 76914 SHA1 hashes now checked by rclone core 76915 .RE 76916 .IP \[bu] 2 76917 Drive 76918 .RS 2 76919 .IP \[bu] 2 76920 Add \f[C]--drive-auth-owner-only\f[R] to only consider files owned by 76921 the user - thanks Bj\[:o]rn Harrtell 76922 .IP \[bu] 2 76923 Export Google documents 76924 .RE 76925 .IP \[bu] 2 76926 Dropbox 76927 .RS 2 76928 .IP \[bu] 2 76929 Make file exclusion error controllable with -q 76930 .RE 76931 .IP \[bu] 2 76932 Swift 76933 .RS 2 76934 .IP \[bu] 2 76935 Fix upload from unprivileged user. 76936 .RE 76937 .IP \[bu] 2 76938 S3 76939 .RS 2 76940 .IP \[bu] 2 76941 Fix updating of mod times of files with \f[C]+\f[R] in. 76942 .RE 76943 .IP \[bu] 2 76944 Local 76945 .RS 2 76946 .IP \[bu] 2 76947 Add local file system option to disable UNC on Windows. 76948 .RE 76949 .SS v1.26 - 2016-01-02 76950 .IP \[bu] 2 76951 New Features 76952 .RS 2 76953 .IP \[bu] 2 76954 Yandex storage backend - thank you Dmitry Burdeev (\[dq]dibu\[dq]) 76955 .IP \[bu] 2 76956 Implement Backblaze B2 storage backend 76957 .IP \[bu] 2 76958 Add --min-age and --max-age flags - thank you Adriano Aur\['e]lio 76959 Meirelles 76960 .IP \[bu] 2 76961 Make ls/lsl/md5sum/size/check obey includes and excludes 76962 .RE 76963 .IP \[bu] 2 76964 Fixes 76965 .RS 2 76966 .IP \[bu] 2 76967 Fix crash in http logging 76968 .IP \[bu] 2 76969 Upload releases to github too 76970 .RE 76971 .IP \[bu] 2 76972 Swift 76973 .RS 2 76974 .IP \[bu] 2 76975 Fix sync for chunked files 76976 .RE 76977 .IP \[bu] 2 76978 OneDrive 76979 .RS 2 76980 .IP \[bu] 2 76981 Re-enable server-side copy 76982 .IP \[bu] 2 76983 Don\[aq]t mask HTTP error codes with JSON decode error 76984 .RE 76985 .IP \[bu] 2 76986 S3 76987 .RS 2 76988 .IP \[bu] 2 76989 Fix corrupting Content-Type on mod time update (thanks Joseph Spurrier) 76990 .RE 76991 .SS v1.25 - 2015-11-14 76992 .IP \[bu] 2 76993 New features 76994 .RS 2 76995 .IP \[bu] 2 76996 Implement Hubic storage system 76997 .RE 76998 .IP \[bu] 2 76999 Fixes 77000 .RS 2 77001 .IP \[bu] 2 77002 Fix deletion of some excluded files without --delete-excluded 77003 .RS 2 77004 .IP \[bu] 2 77005 This could have deleted files unexpectedly on sync 77006 .IP \[bu] 2 77007 Always check first with \f[C]--dry-run\f[R]! 77008 .RE 77009 .RE 77010 .IP \[bu] 2 77011 Swift 77012 .RS 2 77013 .IP \[bu] 2 77014 Stop SetModTime losing metadata (e.g. 77015 X-Object-Manifest) 77016 .RS 2 77017 .IP \[bu] 2 77018 This could have caused data loss for files > 5GB in size 77019 .RE 77020 .IP \[bu] 2 77021 Use ContentType from Object to avoid lookups in listings 77022 .RE 77023 .IP \[bu] 2 77024 OneDrive 77025 .RS 2 77026 .IP \[bu] 2 77027 disable server-side copy as it seems to be broken at Microsoft 77028 .RE 77029 .SS v1.24 - 2015-11-07 77030 .IP \[bu] 2 77031 New features 77032 .RS 2 77033 .IP \[bu] 2 77034 Add support for Microsoft OneDrive 77035 .IP \[bu] 2 77036 Add \f[C]--no-check-certificate\f[R] option to disable server 77037 certificate verification 77038 .IP \[bu] 2 77039 Add async readahead buffer for faster transfer of big files 77040 .RE 77041 .IP \[bu] 2 77042 Fixes 77043 .RS 2 77044 .IP \[bu] 2 77045 Allow spaces in remotes and check remote names for validity at creation 77046 time 77047 .IP \[bu] 2 77048 Allow \[aq]&\[aq] and disallow \[aq]:\[aq] in Windows filenames. 77049 .RE 77050 .IP \[bu] 2 77051 Swift 77052 .RS 2 77053 .IP \[bu] 2 77054 Ignore directory marker objects where appropriate - allows working with 77055 Hubic 77056 .IP \[bu] 2 77057 Don\[aq]t delete the container if fs wasn\[aq]t at root 77058 .RE 77059 .IP \[bu] 2 77060 S3 77061 .RS 2 77062 .IP \[bu] 2 77063 Don\[aq]t delete the bucket if fs wasn\[aq]t at root 77064 .RE 77065 .IP \[bu] 2 77066 Google Cloud Storage 77067 .RS 2 77068 .IP \[bu] 2 77069 Don\[aq]t delete the bucket if fs wasn\[aq]t at root 77070 .RE 77071 .SS v1.23 - 2015-10-03 77072 .IP \[bu] 2 77073 New features 77074 .RS 2 77075 .IP \[bu] 2 77076 Implement \f[C]rclone size\f[R] for measuring remotes 77077 .RE 77078 .IP \[bu] 2 77079 Fixes 77080 .RS 2 77081 .IP \[bu] 2 77082 Fix headless config for drive and gcs 77083 .IP \[bu] 2 77084 Tell the user they should try again if the webserver method failed 77085 .IP \[bu] 2 77086 Improve output of \f[C]--dump-headers\f[R] 77087 .RE 77088 .IP \[bu] 2 77089 S3 77090 .RS 2 77091 .IP \[bu] 2 77092 Allow anonymous access to public buckets 77093 .RE 77094 .IP \[bu] 2 77095 Swift 77096 .RS 2 77097 .IP \[bu] 2 77098 Stop chunked operations logging \[dq]Failed to read info: Object Not 77099 Found\[dq] 77100 .IP \[bu] 2 77101 Use Content-Length on uploads for extra reliability 77102 .RE 77103 .SS v1.22 - 2015-09-28 77104 .IP \[bu] 2 77105 Implement rsync like include and exclude flags 77106 .IP \[bu] 2 77107 swift 77108 .RS 2 77109 .IP \[bu] 2 77110 Support files > 5GB - thanks Sergey Tolmachev 77111 .RE 77112 .SS v1.21 - 2015-09-22 77113 .IP \[bu] 2 77114 New features 77115 .RS 2 77116 .IP \[bu] 2 77117 Display individual transfer progress 77118 .IP \[bu] 2 77119 Make lsl output times in localtime 77120 .RE 77121 .IP \[bu] 2 77122 Fixes 77123 .RS 2 77124 .IP \[bu] 2 77125 Fix allowing user to override credentials again in Drive, GCS and ACD 77126 .RE 77127 .IP \[bu] 2 77128 Amazon Drive 77129 .RS 2 77130 .IP \[bu] 2 77131 Implement compliant pacing scheme 77132 .RE 77133 .IP \[bu] 2 77134 Google Drive 77135 .RS 2 77136 .IP \[bu] 2 77137 Make directory reads concurrent for increased speed. 77138 .RE 77139 .SS v1.20 - 2015-09-15 77140 .IP \[bu] 2 77141 New features 77142 .RS 2 77143 .IP \[bu] 2 77144 Amazon Drive support 77145 .IP \[bu] 2 77146 Oauth support redone - fix many bugs and improve usability 77147 .RS 2 77148 .IP \[bu] 2 77149 Use \[dq]golang.org/x/oauth2\[dq] as oauth library of choice 77150 .IP \[bu] 2 77151 Improve oauth usability for smoother initial signup 77152 .IP \[bu] 2 77153 drive, googlecloudstorage: optionally use auto config for the oauth 77154 token 77155 .RE 77156 .IP \[bu] 2 77157 Implement --dump-headers and --dump-bodies debug flags 77158 .IP \[bu] 2 77159 Show multiple matched commands if abbreviation too short 77160 .IP \[bu] 2 77161 Implement server-side move where possible 77162 .RE 77163 .IP \[bu] 2 77164 local 77165 .RS 2 77166 .IP \[bu] 2 77167 Always use UNC paths internally on Windows - fixes a lot of bugs 77168 .RE 77169 .IP \[bu] 2 77170 dropbox 77171 .RS 2 77172 .IP \[bu] 2 77173 force use of our custom transport which makes timeouts work 77174 .RE 77175 .IP \[bu] 2 77176 Thanks to Klaus Post for lots of help with this release 77177 .SS v1.19 - 2015-08-28 77178 .IP \[bu] 2 77179 New features 77180 .RS 2 77181 .IP \[bu] 2 77182 Server side copies for s3/swift/drive/dropbox/gcs 77183 .IP \[bu] 2 77184 Move command - uses server-side copies if it can 77185 .IP \[bu] 2 77186 Implement --retries flag - tries 3 times by default 77187 .IP \[bu] 2 77188 Build for plan9/amd64 and solaris/amd64 too 77189 .RE 77190 .IP \[bu] 2 77191 Fixes 77192 .RS 2 77193 .IP \[bu] 2 77194 Make a current version download with a fixed URL for scripting 77195 .IP \[bu] 2 77196 Ignore rmdir in limited fs rather than throwing error 77197 .RE 77198 .IP \[bu] 2 77199 dropbox 77200 .RS 2 77201 .IP \[bu] 2 77202 Increase chunk size to improve upload speeds massively 77203 .IP \[bu] 2 77204 Issue an error message when trying to upload bad file name 77205 .RE 77206 .SS v1.18 - 2015-08-17 77207 .IP \[bu] 2 77208 drive 77209 .RS 2 77210 .IP \[bu] 2 77211 Add \f[C]--drive-use-trash\f[R] flag so rclone trashes instead of 77212 deletes 77213 .IP \[bu] 2 77214 Add \[dq]Forbidden to download\[dq] message for files with no 77215 downloadURL 77216 .RE 77217 .IP \[bu] 2 77218 dropbox 77219 .RS 2 77220 .IP \[bu] 2 77221 Remove datastore 77222 .RS 2 77223 .IP \[bu] 2 77224 This was deprecated and it caused a lot of problems 77225 .IP \[bu] 2 77226 Modification times and MD5SUMs no longer stored 77227 .RE 77228 .IP \[bu] 2 77229 Fix uploading files > 2GB 77230 .RE 77231 .IP \[bu] 2 77232 s3 77233 .RS 2 77234 .IP \[bu] 2 77235 use official AWS SDK from github.com/aws/aws-sdk-go 77236 .IP \[bu] 2 77237 \f[B]NB\f[R] will most likely require you to delete and recreate remote 77238 .IP \[bu] 2 77239 enable multipart upload which enables files > 5GB 77240 .IP \[bu] 2 77241 tested with Ceph / RadosGW / S3 emulation 77242 .IP \[bu] 2 77243 many thanks to Sam Liston and Brian Haymore at the Utah Center for High 77244 Performance Computing (https://www.chpc.utah.edu/) for a Ceph test 77245 account 77246 .RE 77247 .IP \[bu] 2 77248 misc 77249 .RS 2 77250 .IP \[bu] 2 77251 Show errors when reading the config file 77252 .IP \[bu] 2 77253 Do not print stats in quiet mode - thanks Leonid Shalupov 77254 .IP \[bu] 2 77255 Add FAQ 77256 .IP \[bu] 2 77257 Fix created directories not obeying umask 77258 .IP \[bu] 2 77259 Linux installation instructions - thanks Shimon Doodkin 77260 .RE 77261 .SS v1.17 - 2015-06-14 77262 .IP \[bu] 2 77263 dropbox: fix case insensitivity issues - thanks Leonid Shalupov 77264 .SS v1.16 - 2015-06-09 77265 .IP \[bu] 2 77266 Fix uploading big files which was causing timeouts or panics 77267 .IP \[bu] 2 77268 Don\[aq]t check md5sum after download with --size-only 77269 .SS v1.15 - 2015-06-06 77270 .IP \[bu] 2 77271 Add --checksum flag to only discard transfers by MD5SUM - thanks Alex 77272 Couper 77273 .IP \[bu] 2 77274 Implement --size-only flag to sync on size not checksum & modtime 77275 .IP \[bu] 2 77276 Expand docs and remove duplicated information 77277 .IP \[bu] 2 77278 Document rclone\[aq]s limitations with directories 77279 .IP \[bu] 2 77280 dropbox: update docs about case insensitivity 77281 .SS v1.14 - 2015-05-21 77282 .IP \[bu] 2 77283 local: fix encoding of non utf-8 file names - fixes a duplicate file 77284 problem 77285 .IP \[bu] 2 77286 drive: docs about rate limiting 77287 .IP \[bu] 2 77288 google cloud storage: Fix compile after API change in 77289 \[dq]google.golang.org/api/storage/v1\[dq] 77290 .SS v1.13 - 2015-05-10 77291 .IP \[bu] 2 77292 Revise documentation (especially sync) 77293 .IP \[bu] 2 77294 Implement --timeout and --conntimeout 77295 .IP \[bu] 2 77296 s3: ignore etags from multipart uploads which aren\[aq]t md5sums 77297 .SS v1.12 - 2015-03-15 77298 .IP \[bu] 2 77299 drive: Use chunked upload for files above a certain size 77300 .IP \[bu] 2 77301 drive: add --drive-chunk-size and --drive-upload-cutoff parameters 77302 .IP \[bu] 2 77303 drive: switch to insert from update when a failed copy deletes the 77304 upload 77305 .IP \[bu] 2 77306 core: Log duplicate files if they are detected 77307 .SS v1.11 - 2015-03-04 77308 .IP \[bu] 2 77309 swift: add region parameter 77310 .IP \[bu] 2 77311 drive: fix crash on failed to update remote mtime 77312 .IP \[bu] 2 77313 In remote paths, change native directory separators to / 77314 .IP \[bu] 2 77315 Add synchronization to ls/lsl/lsd output to stop corruptions 77316 .IP \[bu] 2 77317 Ensure all stats/log messages to go stderr 77318 .IP \[bu] 2 77319 Add --log-file flag to log everything (including panics) to file 77320 .IP \[bu] 2 77321 Make it possible to disable stats printing with --stats=0 77322 .IP \[bu] 2 77323 Implement --bwlimit to limit data transfer bandwidth 77324 .SS v1.10 - 2015-02-12 77325 .IP \[bu] 2 77326 s3: list an unlimited number of items 77327 .IP \[bu] 2 77328 Fix getting stuck in the configurator 77329 .SS v1.09 - 2015-02-07 77330 .IP \[bu] 2 77331 windows: Stop drive letters (e.g. 77332 C:) getting mixed up with remotes (e.g. 77333 drive:) 77334 .IP \[bu] 2 77335 local: Fix directory separators on Windows 77336 .IP \[bu] 2 77337 drive: fix rate limit exceeded errors 77338 .SS v1.08 - 2015-02-04 77339 .IP \[bu] 2 77340 drive: fix subdirectory listing to not list entire drive 77341 .IP \[bu] 2 77342 drive: Fix SetModTime 77343 .IP \[bu] 2 77344 dropbox: adapt code to recent library changes 77345 .SS v1.07 - 2014-12-23 77346 .IP \[bu] 2 77347 google cloud storage: fix memory leak 77348 .SS v1.06 - 2014-12-12 77349 .IP \[bu] 2 77350 Fix \[dq]Couldn\[aq]t find home directory\[dq] on OSX 77351 .IP \[bu] 2 77352 swift: Add tenant parameter 77353 .IP \[bu] 2 77354 Use new location of Google API packages 77355 .SS v1.05 - 2014-08-09 77356 .IP \[bu] 2 77357 Improved tests and consequently lots of minor fixes 77358 .IP \[bu] 2 77359 core: Fix race detected by go race detector 77360 .IP \[bu] 2 77361 core: Fixes after running errcheck 77362 .IP \[bu] 2 77363 drive: reset root directory on Rmdir and Purge 77364 .IP \[bu] 2 77365 fs: Document that Purger returns error on empty directory, test and fix 77366 .IP \[bu] 2 77367 google cloud storage: fix ListDir on subdirectory 77368 .IP \[bu] 2 77369 google cloud storage: re-read metadata in SetModTime 77370 .IP \[bu] 2 77371 s3: make reading metadata more reliable to work around eventual 77372 consistency problems 77373 .IP \[bu] 2 77374 s3: strip trailing / from ListDir() 77375 .IP \[bu] 2 77376 swift: return directories without / in ListDir 77377 .SS v1.04 - 2014-07-21 77378 .IP \[bu] 2 77379 google cloud storage: Fix crash on Update 77380 .SS v1.03 - 2014-07-20 77381 .IP \[bu] 2 77382 swift, s3, dropbox: fix updated files being marked as corrupted 77383 .IP \[bu] 2 77384 Make compile with go 1.1 again 77385 .SS v1.02 - 2014-07-19 77386 .IP \[bu] 2 77387 Implement Dropbox remote 77388 .IP \[bu] 2 77389 Implement Google Cloud Storage remote 77390 .IP \[bu] 2 77391 Verify Md5sums and Sizes after copies 77392 .IP \[bu] 2 77393 Remove times from \[dq]ls\[dq] command - lists sizes only 77394 .IP \[bu] 2 77395 Add add \[dq]lsl\[dq] - lists times and sizes 77396 .IP \[bu] 2 77397 Add \[dq]md5sum\[dq] command 77398 .SS v1.01 - 2014-07-04 77399 .IP \[bu] 2 77400 drive: fix transfer of big files using up lots of memory 77401 .SS v1.00 - 2014-07-03 77402 .IP \[bu] 2 77403 drive: fix whole second dates 77404 .SS v0.99 - 2014-06-26 77405 .IP \[bu] 2 77406 Fix --dry-run not working 77407 .IP \[bu] 2 77408 Make compatible with go 1.1 77409 .SS v0.98 - 2014-05-30 77410 .IP \[bu] 2 77411 s3: Treat missing Content-Length as 0 for some ceph installations 77412 .IP \[bu] 2 77413 rclonetest: add file with a space in 77414 .SS v0.97 - 2014-05-05 77415 .IP \[bu] 2 77416 Implement copying of single files 77417 .IP \[bu] 2 77418 s3 & swift: support paths inside containers/buckets 77419 .SS v0.96 - 2014-04-24 77420 .IP \[bu] 2 77421 drive: Fix multiple files of same name being created 77422 .IP \[bu] 2 77423 drive: Use o.Update and fs.Put to optimise transfers 77424 .IP \[bu] 2 77425 Add version number, -V and --version 77426 .SS v0.95 - 2014-03-28 77427 .IP \[bu] 2 77428 rclone.org: website, docs and graphics 77429 .IP \[bu] 2 77430 drive: fix path parsing 77431 .SS v0.94 - 2014-03-27 77432 .IP \[bu] 2 77433 Change remote format one last time 77434 .IP \[bu] 2 77435 GNU style flags 77436 .SS v0.93 - 2014-03-16 77437 .IP \[bu] 2 77438 drive: store token in config file 77439 .IP \[bu] 2 77440 cross compile other versions 77441 .IP \[bu] 2 77442 set strict permissions on config file 77443 .SS v0.92 - 2014-03-15 77444 .IP \[bu] 2 77445 Config fixes and --config option 77446 .SS v0.91 - 2014-03-15 77447 .IP \[bu] 2 77448 Make config file 77449 .SS v0.90 - 2013-06-27 77450 .IP \[bu] 2 77451 Project named rclone 77452 .SS v0.00 - 2012-11-18 77453 .IP \[bu] 2 77454 Project started 77455 .SH Bugs and Limitations 77456 .SS Limitations 77457 .SS Directory timestamps aren\[aq]t preserved on some backends 77458 .PP 77459 As of \f[C]v1.66\f[R], rclone supports syncing directory modtimes, if 77460 the backend supports it. 77461 Some backends do not support it -- see 77462 overview (https://rclone.org/overview/) for a complete list. 77463 Additionally, note that empty directories are not synced by default 77464 (this can be enabled with \f[C]--create-empty-src-dirs\f[R].) 77465 .SS Rclone struggles with millions of files in a directory/bucket 77466 .PP 77467 Currently rclone loads each directory/bucket entirely into memory before 77468 using it. 77469 Since each rclone object takes 0.5k-1k of memory this can take a very 77470 long time and use a large amount of memory. 77471 .PP 77472 Millions of files in a directory tends to occur on bucket-based remotes 77473 (e.g. 77474 S3 buckets) since those remotes do not segregate subdirectories within 77475 the bucket. 77476 .SS Bucket-based remotes and folders 77477 .PP 77478 Bucket-based remotes (e.g. 77479 S3/GCS/Swift/B2) do not have a concept of directories. 77480 Rclone therefore cannot create directories in them which means that 77481 empty directories on a bucket-based remote will tend to disappear. 77482 .PP 77483 Some software creates empty keys ending in \f[C]/\f[R] as directory 77484 markers. 77485 Rclone doesn\[aq]t do this as it potentially creates more objects and 77486 costs more. 77487 This ability may be added in the future (probably via a flag/option). 77488 .SS Bugs 77489 .PP 77490 Bugs are stored in rclone\[aq]s GitHub project: 77491 .IP \[bu] 2 77492 Reported 77493 bugs (https://github.com/artpar/artpar/issues?q=is%3Aopen+is%3Aissue+label%3Abug) 77494 .IP \[bu] 2 77495 Known 77496 issues (https://github.com/artpar/artpar/issues?q=is%3Aopen+is%3Aissue+milestone%3A%22Known+Problem%22) 77497 .SH Frequently Asked Questions 77498 .SS Do all cloud storage systems support all rclone commands 77499 .PP 77500 Yes they do. 77501 All the rclone commands (e.g. 77502 \f[C]sync\f[R], \f[C]copy\f[R], etc.) will work on all the remote 77503 storage systems. 77504 .SS Can I copy the config from one machine to another 77505 .PP 77506 Sure! Rclone stores all of its config in a single file. 77507 If you want to find this file, run \f[C]rclone config file\f[R] which 77508 will tell you where it is. 77509 .PP 77510 See the remote setup docs (https://rclone.org/remote_setup/) for more 77511 info. 77512 .SS How do I configure rclone on a remote / headless box with no browser? 77513 .PP 77514 This has now been documented in its own remote setup 77515 page (https://rclone.org/remote_setup/). 77516 .SS Can rclone sync directly from drive to s3 77517 .PP 77518 Rclone can sync between two remote cloud storage systems just fine. 77519 .PP 77520 Note that it effectively downloads the file and uploads it again, so the 77521 node running rclone would need to have lots of bandwidth. 77522 .PP 77523 The syncs would be incremental (on a file by file basis). 77524 .PP 77525 e.g. 77526 .IP 77527 .nf 77528 \f[C] 77529 rclone sync --interactive drive:Folder s3:bucket 77530 \f[R] 77531 .fi 77532 .SS Using rclone from multiple locations at the same time 77533 .PP 77534 You can use rclone from multiple places at the same time if you choose 77535 different subdirectory for the output, e.g. 77536 .IP 77537 .nf 77538 \f[C] 77539 Server A> rclone sync --interactive /tmp/whatever remote:ServerA 77540 Server B> rclone sync --interactive /tmp/whatever remote:ServerB 77541 \f[R] 77542 .fi 77543 .PP 77544 If you sync to the same directory then you should use rclone copy 77545 otherwise the two instances of rclone may delete each other\[aq]s files, 77546 e.g. 77547 .IP 77548 .nf 77549 \f[C] 77550 Server A> rclone copy /tmp/whatever remote:Backup 77551 Server B> rclone copy /tmp/whatever remote:Backup 77552 \f[R] 77553 .fi 77554 .PP 77555 The file names you upload from Server A and Server B should be different 77556 in this case, otherwise some file systems (e.g. 77557 Drive) may make duplicates. 77558 .SS Why doesn\[aq]t rclone support partial transfers / binary diffs like rsync? 77559 .PP 77560 Rclone stores each file you transfer as a native object on the remote 77561 cloud storage system. 77562 This means that you can see the files you upload as expected using 77563 alternative access methods (e.g. 77564 using the Google Drive web interface). 77565 There is a 1:1 mapping between files on your hard disk and objects 77566 created in the cloud storage system. 77567 .PP 77568 Cloud storage systems (at least none I\[aq]ve come across yet) don\[aq]t 77569 support partially uploading an object. 77570 You can\[aq]t take an existing object, and change some bytes in the 77571 middle of it. 77572 .PP 77573 It would be possible to make a sync system which stored binary diffs 77574 instead of whole objects like rclone does, but that would break the 1:1 77575 mapping of files on your hard disk to objects in the remote cloud 77576 storage system. 77577 .PP 77578 All the cloud storage systems support partial downloads of content, so 77579 it would be possible to make partial downloads work. 77580 However to make this work efficiently this would require storing a 77581 significant amount of metadata, which breaks the desired 1:1 mapping of 77582 files to objects. 77583 .SS Can rclone do bi-directional sync? 77584 .PP 77585 Yes, since rclone v1.58.0, bidirectional cloud 77586 sync (https://rclone.org/bisync/) is available. 77587 .SS Can I use rclone with an HTTP proxy? 77588 .PP 77589 Yes. 77590 rclone will follow the standard environment variables for proxies, 77591 similar to cURL and other programs. 77592 .PP 77593 In general the variables are called \f[C]http_proxy\f[R] (for services 77594 reached over \f[C]http\f[R]) and \f[C]https_proxy\f[R] (for services 77595 reached over \f[C]https\f[R]). 77596 Most public services will be using \f[C]https\f[R], but you may wish to 77597 set both. 77598 .PP 77599 The content of the variable is \f[C]protocol://server:port\f[R]. 77600 The protocol value is the one used to talk to the proxy server, itself, 77601 and is commonly either \f[C]http\f[R] or \f[C]socks5\f[R]. 77602 .PP 77603 Slightly annoyingly, there is no \f[I]standard\f[R] for the name; some 77604 applications may use \f[C]http_proxy\f[R] but another one 77605 \f[C]HTTP_PROXY\f[R]. 77606 The \f[C]Go\f[R] libraries used by \f[C]rclone\f[R] will try both 77607 variations, but you may wish to set all possibilities. 77608 So, on Linux, you may end up with code similar to 77609 .IP 77610 .nf 77611 \f[C] 77612 export http_proxy=http://proxyserver:12345 77613 export https_proxy=$http_proxy 77614 export HTTP_PROXY=$http_proxy 77615 export HTTPS_PROXY=$http_proxy 77616 \f[R] 77617 .fi 77618 .PP 77619 Note: If the proxy server requires a username and password, then use 77620 .IP 77621 .nf 77622 \f[C] 77623 export http_proxy=http://username:password\[at]proxyserver:12345 77624 export https_proxy=$http_proxy 77625 export HTTP_PROXY=$http_proxy 77626 export HTTPS_PROXY=$http_proxy 77627 \f[R] 77628 .fi 77629 .PP 77630 The \f[C]NO_PROXY\f[R] allows you to disable the proxy for specific 77631 hosts. 77632 Hosts must be comma separated, and can contain domains or parts. 77633 For instance \[dq]foo.com\[dq] also matches \[dq]bar.foo.com\[dq]. 77634 .PP 77635 e.g. 77636 .IP 77637 .nf 77638 \f[C] 77639 export no_proxy=localhost,127.0.0.0/8,my.host.name 77640 export NO_PROXY=$no_proxy 77641 \f[R] 77642 .fi 77643 .PP 77644 Note that the FTP backend does not support \f[C]ftp_proxy\f[R] yet. 77645 .SS Rclone gives x509: failed to load system roots and no roots provided error 77646 .PP 77647 This means that \f[C]rclone\f[R] can\[aq]t find the SSL root 77648 certificates. 77649 Likely you are running \f[C]rclone\f[R] on a NAS with a cut-down Linux 77650 OS, or possibly on Solaris. 77651 .PP 77652 Rclone (via the Go runtime) tries to load the root certificates from 77653 these places on Linux. 77654 .IP 77655 .nf 77656 \f[C] 77657 \[dq]/etc/ssl/certs/ca-certificates.crt\[dq], // Debian/Ubuntu/Gentoo etc. 77658 \[dq]/etc/pki/tls/certs/ca-bundle.crt\[dq], // Fedora/RHEL 77659 \[dq]/etc/ssl/ca-bundle.pem\[dq], // OpenSUSE 77660 \[dq]/etc/pki/tls/cacert.pem\[dq], // OpenELEC 77661 \f[R] 77662 .fi 77663 .PP 77664 So doing something like this should fix the problem. 77665 It also sets the time which is important for SSL to work properly. 77666 .IP 77667 .nf 77668 \f[C] 77669 mkdir -p /etc/ssl/certs/ 77670 curl -o /etc/ssl/certs/ca-certificates.crt https://raw.githubusercontent.com/bagder/ca-bundle/master/ca-bundle.crt 77671 ntpclient -s -h pool.ntp.org 77672 \f[R] 77673 .fi 77674 .PP 77675 The two environment variables \f[C]SSL_CERT_FILE\f[R] and 77676 \f[C]SSL_CERT_DIR\f[R], mentioned in the x509 77677 package (https://godoc.org/crypto/x509), provide an additional way to 77678 provide the SSL root certificates. 77679 .PP 77680 Note that you may need to add the \f[C]--insecure\f[R] option to the 77681 \f[C]curl\f[R] command line if it doesn\[aq]t work without. 77682 .IP 77683 .nf 77684 \f[C] 77685 curl --insecure -o /etc/ssl/certs/ca-certificates.crt https://raw.githubusercontent.com/bagder/ca-bundle/master/ca-bundle.crt 77686 \f[R] 77687 .fi 77688 .SS Rclone gives Failed to load config file: function not implemented error 77689 .PP 77690 Likely this means that you are running rclone on Linux version not 77691 supported by the go runtime, ie earlier than version 2.6.23. 77692 .PP 77693 See the system requirements section in the go install 77694 docs (https://golang.org/doc/install) for full details. 77695 .SS All my uploaded docx/xlsx/pptx files appear as archive/zip 77696 .PP 77697 This is caused by uploading these files from a Windows computer which 77698 hasn\[aq]t got the Microsoft Office suite installed. 77699 The easiest way to fix is to install the Word viewer and the Microsoft 77700 Office Compatibility Pack for Word, Excel, and PowerPoint 2007 and later 77701 versions\[aq] file formats 77702 .SS tcp lookup some.domain.com no such host 77703 .PP 77704 This happens when rclone cannot resolve a domain. 77705 Please check that your DNS setup is generally working, e.g. 77706 .IP 77707 .nf 77708 \f[C] 77709 # both should print a long list of possible IP addresses 77710 dig www.googleapis.com # resolve using your default DNS 77711 dig www.googleapis.com \[at]8.8.8.8 # resolve with Google\[aq]s DNS server 77712 \f[R] 77713 .fi 77714 .PP 77715 If you are using \f[C]systemd-resolved\f[R] (default on Arch Linux), 77716 ensure it is at version 233 or higher. 77717 Previous releases contain a bug which causes not all domains to be 77718 resolved properly. 77719 .PP 77720 The Go resolver decision can be influenced with the 77721 \f[C]GODEBUG=netdns=...\f[R] environment variable. 77722 This also allows to resolve certain issues with DNS resolution. 77723 On Windows or MacOS systems, try forcing use of the internal Go resolver 77724 by setting \f[C]GODEBUG=netdns=go\f[R] at runtime. 77725 On other systems (Linux, *BSD, etc) try forcing use of the system name 77726 resolver by setting \f[C]GODEBUG=netdns=cgo\f[R] (and recompile rclone 77727 from source with CGO enabled if necessary). 77728 See the name resolution section in the go 77729 docs (https://golang.org/pkg/net/#hdr-Name_Resolution). 77730 .SS Failed to start auth webserver on Windows 77731 .IP 77732 .nf 77733 \f[C] 77734 Error: config failed to refresh token: failed to start auth webserver: listen tcp 127.0.0.1:53682: bind: An attempt was made to access a socket in a way forbidden by its access permissions. 77735 \&... 77736 yyyy/mm/dd hh:mm:ss Fatal error: config failed to refresh token: failed to start auth webserver: listen tcp 127.0.0.1:53682: bind: An attempt was made to access a socket in a way forbidden by its access permissions. 77737 \f[R] 77738 .fi 77739 .PP 77740 This is sometimes caused by the Host Network Service causing issues with 77741 opening the port on the host. 77742 .PP 77743 A simple solution may be restarting the Host Network Service with eg. 77744 Powershell 77745 .IP 77746 .nf 77747 \f[C] 77748 Restart-Service hns 77749 \f[R] 77750 .fi 77751 .SS The total size reported in the stats for a sync is wrong and keeps changing 77752 .PP 77753 It is likely you have more than 10,000 files that need to be synced. 77754 By default, rclone only gets 10,000 files ahead in a sync so as not to 77755 use up too much memory. 77756 You can change this default with the 77757 --max-backlog (https://rclone.org/docs/#max-backlog-n) flag. 77758 .SS Rclone is using too much memory or appears to have a memory leak 77759 .PP 77760 Rclone is written in Go which uses a garbage collector. 77761 The default settings for the garbage collector mean that it runs when 77762 the heap size has doubled. 77763 .PP 77764 However it is possible to tune the garbage collector to use less memory 77765 by setting GOGC (https://dave.cheney.net/tag/gogc) to a lower value, say 77766 \f[C]export GOGC=20\f[R]. 77767 This will make the garbage collector work harder, reducing memory size 77768 at the expense of CPU usage. 77769 .PP 77770 The most common cause of rclone using lots of memory is a single 77771 directory with thousands or millions of files in. 77772 Rclone has to load this entirely into memory as rclone objects. 77773 Each rclone object takes 0.5k-1k of memory. 77774 .SS Rclone changes fullwidth Unicode punctuation marks in file names 77775 .PP 77776 For example: On a Windows system, you have a file with name 77777 \f[C]Test\[uFF1A]1.jpg\f[R], where \f[C]\[uFF1A]\f[R] is the Unicode 77778 fullwidth colon symbol. 77779 When using rclone to copy this to your Google Drive, you will notice 77780 that the file gets renamed to \f[C]Test:1.jpg\f[R], where \f[C]:\f[R] is 77781 the regular (halfwidth) colon. 77782 .PP 77783 The reason for such renames is the way rclone handles different 77784 restricted filenames (https://rclone.org/overview/#restricted-filenames) 77785 on different cloud storage systems. 77786 It tries to avoid ambiguous file names as much and allow moving files 77787 between many cloud storage systems transparently, by replacing invalid 77788 characters with similar looking Unicode characters when transferring to 77789 one storage system, and replacing back again when transferring to a 77790 different storage system where the original characters are supported. 77791 When the same Unicode characters are intentionally used in file names, 77792 this replacement strategy leads to unwanted renames. 77793 Read more 77794 here (https://rclone.org/overview/#restricted-filenames-caveats). 77795 .SH License 77796 .PP 77797 This is free software under the terms of the MIT license (check the 77798 COPYING file included with the source code). 77799 .IP 77800 .nf 77801 \f[C] 77802 Copyright (C) 2019 by Nick Craig-Wood https://www.craig-wood.com/nick/ 77803 77804 Permission is hereby granted, free of charge, to any person obtaining a copy 77805 of this software and associated documentation files (the \[dq]Software\[dq]), to deal 77806 in the Software without restriction, including without limitation the rights 77807 to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 77808 copies of the Software, and to permit persons to whom the Software is 77809 furnished to do so, subject to the following conditions: 77810 77811 The above copyright notice and this permission notice shall be included in 77812 all copies or substantial portions of the Software. 77813 77814 THE SOFTWARE IS PROVIDED \[dq]AS IS\[dq], WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 77815 IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 77816 FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 77817 AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 77818 LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 77819 OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN 77820 THE SOFTWARE. 77821 \f[R] 77822 .fi 77823 .SH Authors and contributors 77824 .SS Authors 77825 .IP \[bu] 2 77826 Nick Craig-Wood <nick@craig-wood.com> 77827 .SS Contributors 77828 .PP 77829 {{< rem 77830 \f[C]email addresses removed from here need to be added to bin/.ignore-emails to make sure update-authors.py doesn\[aq]t immediately put them back in again.\f[R] 77831 >}} 77832 .IP \[bu] 2 77833 Alex Couper <amcouper@gmail.com> 77834 .IP \[bu] 2 77835 Leonid Shalupov <leonid@shalupov.com> <shalupov@diverse.org.ru> 77836 .IP \[bu] 2 77837 Shimon Doodkin <helpmepro1@gmail.com> 77838 .IP \[bu] 2 77839 Colin Nicholson <colin@colinn.com> 77840 .IP \[bu] 2 77841 Klaus Post <klauspost@gmail.com> 77842 .IP \[bu] 2 77843 Sergey Tolmachev <tolsi.ru@gmail.com> 77844 .IP \[bu] 2 77845 Adriano Aur\['e]lio Meirelles <adriano@atinge.com> 77846 .IP \[bu] 2 77847 C. 77848 Bess <cbess@users.noreply.github.com> 77849 .IP \[bu] 2 77850 Dmitry Burdeev <dibu28@gmail.com> 77851 .IP \[bu] 2 77852 Joseph Spurrier <github@josephspurrier.com> 77853 .IP \[bu] 2 77854 Bj\[:o]rn Harrtell <bjorn@wololo.org> 77855 .IP \[bu] 2 77856 Xavier Lucas <xavier.lucas@corp.ovh.com> 77857 .IP \[bu] 2 77858 Werner Beroux <werner@beroux.com> 77859 .IP \[bu] 2 77860 Brian Stengaard <brian@stengaard.eu> 77861 .IP \[bu] 2 77862 Jakub Gedeon <jgedeon@sofi.com> 77863 .IP \[bu] 2 77864 Jim Tittsler <jwt@onjapan.net> 77865 .IP \[bu] 2 77866 Michal Witkowski <michal@improbable.io> 77867 .IP \[bu] 2 77868 Fabian Ruff <fabian.ruff@sap.com> 77869 .IP \[bu] 2 77870 Leigh Klotz <klotz@quixey.com> 77871 .IP \[bu] 2 77872 Romain Lapray <lapray.romain@gmail.com> 77873 .IP \[bu] 2 77874 Justin R. 77875 Wilson <jrw972@gmail.com> 77876 .IP \[bu] 2 77877 Antonio Messina <antonio.s.messina@gmail.com> 77878 .IP \[bu] 2 77879 Stefan G. 77880 Weichinger <office@oops.co.at> 77881 .IP \[bu] 2 77882 Per Cederberg <cederberg@gmail.com> 77883 .IP \[bu] 2 77884 Radek \[vS]enfeld <rush@logic.cz> 77885 .IP \[bu] 2 77886 Fredrik Fornwall <fredrik@fornwall.net> 77887 .IP \[bu] 2 77888 Asko Tamm <asko@deekit.net> 77889 .IP \[bu] 2 77890 xor-zz <xor@gstocco.com> 77891 .IP \[bu] 2 77892 Tomasz Mazur <tmazur90@gmail.com> 77893 .IP \[bu] 2 77894 Marco Paganini <paganini@paganini.net> 77895 .IP \[bu] 2 77896 Felix B\[:u]nemann <buenemann@louis.info> 77897 .IP \[bu] 2 77898 Durval Menezes <jmrclone@durval.com> 77899 .IP \[bu] 2 77900 Luiz Carlos Rumbelsperger Viana <maxd13_luiz_carlos@hotmail.com> 77901 .IP \[bu] 2 77902 Stefan Breunig <stefan-github@yrden.de> 77903 .IP \[bu] 2 77904 Alishan Ladhani <ali-l@users.noreply.github.com> 77905 .IP \[bu] 2 77906 0xJAKE <0xJAKE@users.noreply.github.com> 77907 .IP \[bu] 2 77908 Thibault Molleman <thibaultmol@users.noreply.github.com> 77909 .IP \[bu] 2 77910 Scott McGillivray <scott.mcgillivray@gmail.com> 77911 .IP \[bu] 2 77912 Bj\[/o]rn Erik Pedersen <bjorn.erik.pedersen@gmail.com> 77913 .IP \[bu] 2 77914 Lukas Loesche <lukas@mesosphere.io> 77915 .IP \[bu] 2 77916 emyarod <emyarod@users.noreply.github.com> 77917 .IP \[bu] 2 77918 T.C. 77919 Ferguson <tcf909@gmail.com> 77920 .IP \[bu] 2 77921 Brandur <brandur@mutelight.org> 77922 .IP \[bu] 2 77923 Dario Giovannetti <dev@dariogiovannetti.net> 77924 .IP \[bu] 2 77925 K\['a]roly Ol\['a]h <okaresz@aol.com> 77926 .IP \[bu] 2 77927 Jon Yergatian <jon@macfanatic.ca> 77928 .IP \[bu] 2 77929 Jack Schmidt <github@mowsey.org> 77930 .IP \[bu] 2 77931 Dedsec1 <Dedsec1@users.noreply.github.com> 77932 .IP \[bu] 2 77933 Hisham Zarka <hzarka@gmail.com> 77934 .IP \[bu] 2 77935 J\['e]r\[^o]me Vizcaino <jerome.vizcaino@gmail.com> 77936 .IP \[bu] 2 77937 Mike Tesch <mjt6129@rit.edu> 77938 .IP \[bu] 2 77939 Marvin Watson <marvwatson@users.noreply.github.com> 77940 .IP \[bu] 2 77941 Danny Tsai <danny8376@gmail.com> 77942 .IP \[bu] 2 77943 Yoni Jah <yonjah+git@gmail.com> <yonjah+github@gmail.com> 77944 .IP \[bu] 2 77945 Stephen Harris <github@spuddy.org> <sweharris@users.noreply.github.com> 77946 .IP \[bu] 2 77947 Ihor Dvoretskyi <ihor.dvoretskyi@gmail.com> 77948 .IP \[bu] 2 77949 Jon Craton <jncraton@gmail.com> 77950 .IP \[bu] 2 77951 Hraban Luyat <hraban@0brg.net> 77952 .IP \[bu] 2 77953 Michael Ledin <mledin89@gmail.com> 77954 .IP \[bu] 2 77955 Martin Kristensen <me@azgul.com> 77956 .IP \[bu] 2 77957 Too Much IO <toomuchio@users.noreply.github.com> 77958 .IP \[bu] 2 77959 Anisse Astier <anisse@astier.eu> 77960 .IP \[bu] 2 77961 Zahiar Ahmed <zahiar@live.com> 77962 .IP \[bu] 2 77963 Igor Kharin <igorkharin@gmail.com> 77964 .IP \[bu] 2 77965 Bill Zissimopoulos <billziss@navimatics.com> 77966 .IP \[bu] 2 77967 Bob Potter <bobby.potter@gmail.com> 77968 .IP \[bu] 2 77969 Steven Lu <tacticalazn@gmail.com> 77970 .IP \[bu] 2 77971 Sjur Fredriksen <sjurtf@ifi.uio.no> 77972 .IP \[bu] 2 77973 Ruwbin <hubus12345@gmail.com> 77974 .IP \[bu] 2 77975 Fabian M\[:o]ller <fabianm88@gmail.com> <f.moeller@nynex.de> 77976 .IP \[bu] 2 77977 Edward Q. 77978 Bridges <github@eqbridges.com> 77979 .IP \[bu] 2 77980 Vasiliy Tolstov <v.tolstov@selfip.ru> 77981 .IP \[bu] 2 77982 Harshavardhana <harsha@minio.io> 77983 .IP \[bu] 2 77984 sainaen <sainaen@gmail.com> 77985 .IP \[bu] 2 77986 gdm85 <gdm85@users.noreply.github.com> 77987 .IP \[bu] 2 77988 Yaroslav Halchenko <debian@onerussian.com> 77989 .IP \[bu] 2 77990 John Papandriopoulos <jpap@users.noreply.github.com> 77991 .IP \[bu] 2 77992 Zhiming Wang <zmwangx@gmail.com> 77993 .IP \[bu] 2 77994 Andy Pilate <cubox@cubox.me> 77995 .IP \[bu] 2 77996 Oliver Heyme <olihey@googlemail.com> <olihey@users.noreply.github.com> 77997 <de8olihe@lego.com> 77998 .IP \[bu] 2 77999 wuyu <wuyu@yunify.com> 78000 .IP \[bu] 2 78001 Andrei Dragomir <adragomi@adobe.com> 78002 .IP \[bu] 2 78003 Christian Br\[:u]ggemann <mail@cbruegg.com> 78004 .IP \[bu] 2 78005 Alex McGrath Kraak <amkdude@gmail.com> 78006 .IP \[bu] 2 78007 bpicode <bjoern.pirnay@googlemail.com> 78008 .IP \[bu] 2 78009 Daniel Jagszent <daniel@jagszent.de> 78010 .IP \[bu] 2 78011 Josiah White <thegenius2009@gmail.com> 78012 .IP \[bu] 2 78013 Ishuah Kariuki <kariuki@ishuah.com> <ishuah91@gmail.com> 78014 .IP \[bu] 2 78015 Jan Varho <jan@varho.org> 78016 .IP \[bu] 2 78017 Girish Ramakrishnan <girish@cloudron.io> 78018 .IP \[bu] 2 78019 LingMan <LingMan@users.noreply.github.com> 78020 .IP \[bu] 2 78021 Jacob McNamee <jacobmcnamee@gmail.com> 78022 .IP \[bu] 2 78023 jersou <jertux@gmail.com> 78024 .IP \[bu] 2 78025 thierry <thierry@substantiel.fr> 78026 .IP \[bu] 2 78027 Simon Leinen <simon.leinen@gmail.com> <ubuntu@s3-test.novalocal> 78028 .IP \[bu] 2 78029 Dan Dascalescu <ddascalescu+github@gmail.com> 78030 .IP \[bu] 2 78031 Jason Rose <jason@jro.io> 78032 .IP \[bu] 2 78033 Andrew Starr-Bochicchio <a.starr.b@gmail.com> 78034 .IP \[bu] 2 78035 John Leach <john@johnleach.co.uk> 78036 .IP \[bu] 2 78037 Corban Raun <craun@instructure.com> 78038 .IP \[bu] 2 78039 Pierre Carlson <mpcarl@us.ibm.com> 78040 .IP \[bu] 2 78041 Ernest Borowski <er.borowski@gmail.com> 78042 .IP \[bu] 2 78043 Remus Bunduc <remus.bunduc@gmail.com> 78044 .IP \[bu] 2 78045 Iakov Davydov <iakov.davydov@unil.ch> <dav05.gith@myths.ru> 78046 .IP \[bu] 2 78047 Jakub Tasiemski <tasiemski@gmail.com> 78048 .IP \[bu] 2 78049 David Minor <dminor@saymedia.com> 78050 .IP \[bu] 2 78051 Tim Cooijmans <cooijmans.tim@gmail.com> 78052 .IP \[bu] 2 78053 Laurence <liuxy6@gmail.com> 78054 .IP \[bu] 2 78055 Giovanni Pizzi <gio.piz@gmail.com> 78056 .IP \[bu] 2 78057 Filip Bartodziej <filipbartodziej@gmail.com> 78058 .IP \[bu] 2 78059 Jon Fautley <jon@dead.li> 78060 .IP \[bu] 2 78061 lewapm <32110057+lewapm@users.noreply.github.com> 78062 .IP \[bu] 2 78063 Yassine Imounachen <yassine256@gmail.com> 78064 .IP \[bu] 2 78065 Chris Redekop <chris-redekop@users.noreply.github.com> 78066 <chris.redekop@gmail.com> 78067 .IP \[bu] 2 78068 Jon Fautley <jon@adenoid.appstal.co.uk> 78069 .IP \[bu] 2 78070 Will Gunn <WillGunn@users.noreply.github.com> 78071 .IP \[bu] 2 78072 Lucas Bremgartner <lucas@bremis.ch> 78073 .IP \[bu] 2 78074 Jody Frankowski <jody.frankowski@gmail.com> 78075 .IP \[bu] 2 78076 Andreas Roussos <arouss1980@gmail.com> 78077 .IP \[bu] 2 78078 nbuchanan <nbuchanan@utah.gov> 78079 .IP \[bu] 2 78080 Durval Menezes <rclone@durval.com> 78081 .IP \[bu] 2 78082 Victor <vb-github@viblo.se> 78083 .IP \[bu] 2 78084 Mateusz <pabian.mateusz@gmail.com> 78085 .IP \[bu] 2 78086 Daniel Loader <spicypixel@gmail.com> 78087 .IP \[bu] 2 78088 David0rk <davidork@gmail.com> 78089 .IP \[bu] 2 78090 Alexander Neumann <alexander@bumpern.de> 78091 .IP \[bu] 2 78092 Giri Badanahatti <gbadanahatti@us.ibm.com@Giris-MacBook-Pro.local> 78093 .IP \[bu] 2 78094 Leo R. 78095 Lundgren <leo@finalresort.org> 78096 .IP \[bu] 2 78097 wolfv <wolfv6@users.noreply.github.com> 78098 .IP \[bu] 2 78099 Dave Pedu <dave@davepedu.com> 78100 .IP \[bu] 2 78101 Stefan Lindblom <lindblom@spotify.com> 78102 .IP \[bu] 2 78103 seuffert <oliver@seuffert.biz> 78104 .IP \[bu] 2 78105 gbadanahatti <37121690+gbadanahatti@users.noreply.github.com> 78106 .IP \[bu] 2 78107 Keith Goldfarb <barkofdelight@gmail.com> 78108 .IP \[bu] 2 78109 Steve Kriss <steve@heptio.com> 78110 .IP \[bu] 2 78111 Chih-Hsuan Yen <yan12125@gmail.com> 78112 .IP \[bu] 2 78113 Alexander Neumann <fd0@users.noreply.github.com> 78114 .IP \[bu] 2 78115 Matt Holt <mholt@users.noreply.github.com> 78116 .IP \[bu] 2 78117 Eri Bastos <bastos.eri@gmail.com> 78118 .IP \[bu] 2 78119 Michael P. 78120 Dubner <pywebmail@list.ru> 78121 .IP \[bu] 2 78122 Antoine GIRARD <sapk@users.noreply.github.com> 78123 .IP \[bu] 2 78124 Mateusz Piotrowski <mpp302@gmail.com> 78125 .IP \[bu] 2 78126 Animosity022 <animosity22@users.noreply.github.com> 78127 <earl.texter@gmail.com> 78128 .IP \[bu] 2 78129 Peter Baumgartner <pete@lincolnloop.com> 78130 .IP \[bu] 2 78131 Craig Rachel <craig@craigrachel.com> 78132 .IP \[bu] 2 78133 Michael G. 78134 Noll <miguno@users.noreply.github.com> 78135 .IP \[bu] 2 78136 hensur <me@hensur.de> 78137 .IP \[bu] 2 78138 Oliver Heyme <de8olihe@lego.com> 78139 .IP \[bu] 2 78140 Richard Yang <richard@yenforyang.com> 78141 .IP \[bu] 2 78142 Piotr Oleszczyk <piotr.oleszczyk@gmail.com> 78143 .IP \[bu] 2 78144 Rodrigo <rodarima@gmail.com> 78145 .IP \[bu] 2 78146 NoLooseEnds <NoLooseEnds@users.noreply.github.com> 78147 .IP \[bu] 2 78148 Jakub Karlicek <jakub@karlicek.me> 78149 .IP \[bu] 2 78150 John Clayton <john@codemonkeylabs.com> 78151 .IP \[bu] 2 78152 Kasper Byrdal Nielsen <byrdal76@gmail.com> 78153 .IP \[bu] 2 78154 Benjamin Joseph Dag <bjdag1234@users.noreply.github.com> 78155 .IP \[bu] 2 78156 themylogin <themylogin@gmail.com> 78157 .IP \[bu] 2 78158 Onno Zweers <onno.zweers@surfsara.nl> 78159 .IP \[bu] 2 78160 Jasper Lievisse Adriaanse <jasper@humppa.nl> 78161 .IP \[bu] 2 78162 sandeepkru <sandeep.ummadi@gmail.com> 78163 <sandeepkru@users.noreply.github.com> 78164 .IP \[bu] 2 78165 HerrH <atomtigerzoo@users.noreply.github.com> 78166 .IP \[bu] 2 78167 Andrew <4030760+sparkyman215@users.noreply.github.com> 78168 .IP \[bu] 2 78169 dan smith <XX1011@gmail.com> 78170 .IP \[bu] 2 78171 Oleg Kovalov <iamolegkovalov@gmail.com> 78172 .IP \[bu] 2 78173 Ruben Vandamme <github-com-00ff86@vandamme.email> 78174 .IP \[bu] 2 78175 Cnly <minecnly@gmail.com> 78176 .IP \[bu] 2 78177 Andres Alvarez <1671935+kir4h@users.noreply.github.com> 78178 .IP \[bu] 2 78179 reddi1 <xreddi@gmail.com> 78180 .IP \[bu] 2 78181 Matt Tucker <matthewtckr@gmail.com> 78182 .IP \[bu] 2 78183 Sebastian B\[:u]nger <buengese@gmail.com> <buengese@protonmail.com> 78184 .IP \[bu] 2 78185 Martin Polden <mpolden@mpolden.no> 78186 .IP \[bu] 2 78187 Alex Chen <Cnly@users.noreply.github.com> 78188 .IP \[bu] 2 78189 Denis <deniskovpen@gmail.com> 78190 .IP \[bu] 2 78191 bsteiss <35940619+bsteiss@users.noreply.github.com> 78192 .IP \[bu] 2 78193 C\['e]dric Connes <cedric.connes@gmail.com> 78194 .IP \[bu] 2 78195 Dr. 78196 Tobias Quathamer <toddy15@users.noreply.github.com> 78197 .IP \[bu] 2 78198 dcpu <42736967+dcpu@users.noreply.github.com> 78199 .IP \[bu] 2 78200 Sheldon Rupp <me@shel.io> 78201 .IP \[bu] 2 78202 albertony <12441419+albertony@users.noreply.github.com> 78203 .IP \[bu] 2 78204 cron410 <cron410@gmail.com> 78205 .IP \[bu] 2 78206 Anagh Kumar Baranwal <6824881+darthShadow@users.noreply.github.com> 78207 .IP \[bu] 2 78208 Felix Brucker <felix@felixbrucker.com> 78209 .IP \[bu] 2 78210 Santiago Rodr\['i]guez <scollazo@users.noreply.github.com> 78211 .IP \[bu] 2 78212 Craig Miskell <craig.miskell@fluxfederation.com> 78213 .IP \[bu] 2 78214 Antoine GIRARD <sapk@sapk.fr> 78215 .IP \[bu] 2 78216 Joanna Marek <joanna.marek@u2i.com> 78217 .IP \[bu] 2 78218 frenos <frenos@users.noreply.github.com> 78219 .IP \[bu] 2 78220 ssaqua <ssaqua@users.noreply.github.com> 78221 .IP \[bu] 2 78222 xnaas <me@xnaas.info> 78223 .IP \[bu] 2 78224 Frantisek Fuka <fuka@fuxoft.cz> 78225 .IP \[bu] 2 78226 Paul Kohout <pauljkohout@yahoo.com> 78227 .IP \[bu] 2 78228 dcpu <43330287+dcpu@users.noreply.github.com> 78229 .IP \[bu] 2 78230 jackyzy823 <jackyzy823@gmail.com> 78231 .IP \[bu] 2 78232 David Haguenauer <ml@kurokatta.org> 78233 .IP \[bu] 2 78234 teresy <hi.teresy@gmail.com> 78235 .IP \[bu] 2 78236 buergi <patbuergi@gmx.de> 78237 .IP \[bu] 2 78238 Florian Gamboeck <mail@floga.de> 78239 .IP \[bu] 2 78240 Ralf Hemberger <10364191+rhemberger@users.noreply.github.com> 78241 .IP \[bu] 2 78242 Scott Edlund <sedlund@users.noreply.github.com> 78243 .IP \[bu] 2 78244 Erik Swanson <erik@retailnext.net> 78245 .IP \[bu] 2 78246 Jake Coggiano <jake@stripe.com> 78247 .IP \[bu] 2 78248 brused27 <brused27@noemailaddress> 78249 .IP \[bu] 2 78250 Peter Kaminski <kaminski@istori.com> 78251 .IP \[bu] 2 78252 Henry Ptasinski <henry@logout.com> 78253 .IP \[bu] 2 78254 Alexander <kharkovalexander@gmail.com> 78255 .IP \[bu] 2 78256 Garry McNulty <garrmcnu@gmail.com> 78257 .IP \[bu] 2 78258 Mathieu Carbou <mathieu.carbou@gmail.com> 78259 .IP \[bu] 2 78260 Mark Otway <mark@otway.com> 78261 .IP \[bu] 2 78262 William Cocker <37018962+WilliamCocker@users.noreply.github.com> 78263 .IP \[bu] 2 78264 Fran\[,c]ois Leurent <131.js@cloudyks.org> 78265 .IP \[bu] 2 78266 Arkadius Stefanski <arkste@gmail.com> 78267 .IP \[bu] 2 78268 Jay <dev@jaygoel.com> 78269 .IP \[bu] 2 78270 andrea rota <a@xelera.eu> 78271 .IP \[bu] 2 78272 nicolov <nicolov@users.noreply.github.com> 78273 .IP \[bu] 2 78274 Matt Joiner <anacrolix@gmail.com> 78275 .IP \[bu] 2 78276 Dario Guzik <dario@guzik.com.ar> 78277 .IP \[bu] 2 78278 qip <qip@users.noreply.github.com> 78279 .IP \[bu] 2 78280 yair\[at]unicorn <yair@unicorn> 78281 .IP \[bu] 2 78282 Matt Robinson <brimstone@the.narro.ws> 78283 .IP \[bu] 2 78284 kayrus <kay.diam@gmail.com> 78285 .IP \[bu] 2 78286 R\['e]my L\['e]one <remy.leone@gmail.com> 78287 .IP \[bu] 2 78288 Wojciech Smigielski <wojciech.hieronim.smigielski@gmail.com> 78289 .IP \[bu] 2 78290 weetmuts <oehrstroem@gmail.com> 78291 .IP \[bu] 2 78292 Jonathan <vanillajonathan@users.noreply.github.com> 78293 .IP \[bu] 2 78294 James Carpenter <orbsmiv@users.noreply.github.com> 78295 .IP \[bu] 2 78296 Vince <vince0villamora@gmail.com> 78297 .IP \[bu] 2 78298 Nestar47 <47841759+Nestar47@users.noreply.github.com> 78299 .IP \[bu] 2 78300 Six <brbsix@gmail.com> 78301 .IP \[bu] 2 78302 Alexandru Bumbacea <alexandru.bumbacea@booking.com> 78303 .IP \[bu] 2 78304 calisro <robert.calistri@gmail.com> 78305 .IP \[bu] 2 78306 Dr.Rx <david.rey@nventive.com> 78307 .IP \[bu] 2 78308 marcintustin <marcintustin@users.noreply.github.com> 78309 .IP \[bu] 2 78310 jaKa Mo\[u010D]nik <jaka@koofr.net> 78311 .IP \[bu] 2 78312 Fionera <fionera@fionera.de> 78313 .IP \[bu] 2 78314 Dan Walters <dan@walters.io> 78315 .IP \[bu] 2 78316 Danil Semelenov <sgtpep@users.noreply.github.com> 78317 .IP \[bu] 2 78318 xopez <28950736+xopez@users.noreply.github.com> 78319 .IP \[bu] 2 78320 Ben Boeckel <mathstuf@gmail.com> 78321 .IP \[bu] 2 78322 Manu <manu@snapdragon.cc> 78323 .IP \[bu] 2 78324 Kyle E. 78325 Mitchell <kyle@kemitchell.com> 78326 .IP \[bu] 2 78327 Gary Kim <gary@garykim.dev> 78328 .IP \[bu] 2 78329 Jon <jonathn@github.com> 78330 .IP \[bu] 2 78331 Jeff Quinn <jeffrey.quinn@bluevoyant.com> 78332 .IP \[bu] 2 78333 Peter Berbec <peter@berbec.com> 78334 .IP \[bu] 2 78335 didil <1284255+didil@users.noreply.github.com> 78336 .IP \[bu] 2 78337 id01 <gaviniboom@gmail.com> 78338 .IP \[bu] 2 78339 Robert Marko <robimarko@gmail.com> 78340 .IP \[bu] 2 78341 Philip Harvey <32467456+pharveybattelle@users.noreply.github.com> 78342 .IP \[bu] 2 78343 JorisE <JorisE@users.noreply.github.com> 78344 .IP \[bu] 2 78345 garry415 <garry.415@gmail.com> 78346 .IP \[bu] 2 78347 forgems <forgems@gmail.com> 78348 .IP \[bu] 2 78349 Florian Apolloner <florian@apolloner.eu> 78350 .IP \[bu] 2 78351 Aleksandar Jankovi\['c] <office@ajankovic.com> 78352 <ajankovic@users.noreply.github.com> 78353 .IP \[bu] 2 78354 Maran <maran@protonmail.com> 78355 .IP \[bu] 2 78356 nguyenhuuluan434 <nguyenhuuluan434@gmail.com> 78357 .IP \[bu] 2 78358 Laura Hausmann <zotan@zotan.pw> <laura@hausmann.dev> 78359 .IP \[bu] 2 78360 yparitcher <y@paritcher.com> 78361 .IP \[bu] 2 78362 AbelThar <abela.tharen@gmail.com> 78363 .IP \[bu] 2 78364 Matti Niemenmaa <matti.niemenmaa+git@iki.fi> 78365 .IP \[bu] 2 78366 Russell Davis <russelldavis@users.noreply.github.com> 78367 .IP \[bu] 2 78368 Yi FU <yi.fu@tink.se> 78369 .IP \[bu] 2 78370 Paul Millar <paul.millar@desy.de> 78371 .IP \[bu] 2 78372 justinalin <justinalin@qnap.com> 78373 .IP \[bu] 2 78374 EliEron <subanimehd@gmail.com> 78375 .IP \[bu] 2 78376 justina777 <chiahuei.lin@gmail.com> 78377 .IP \[bu] 2 78378 Chaitanya Bankanhal <bchaitanya15@gmail.com> 78379 .IP \[bu] 2 78380 Micha\[/l] Matczuk <michal@scylladb.com> 78381 .IP \[bu] 2 78382 Macavirus <macavirus@zoho.com> 78383 .IP \[bu] 2 78384 Abhinav Sharma <abhi18av@outlook.com> 78385 .IP \[bu] 2 78386 ginvine <34869051+ginvine@users.noreply.github.com> 78387 .IP \[bu] 2 78388 Patrick Wang <mail6543210@yahoo.com.tw> 78389 .IP \[bu] 2 78390 Cenk Alti <cenkalti@gmail.com> 78391 .IP \[bu] 2 78392 Andreas Chlupka <andy@chlupka.com> 78393 .IP \[bu] 2 78394 Alfonso Montero <amontero@tinet.org> 78395 .IP \[bu] 2 78396 Ivan Andreev <ivandeex@gmail.com> 78397 .IP \[bu] 2 78398 David Baumgold <david@davidbaumgold.com> 78399 .IP \[bu] 2 78400 Lars Lehtonen <lars.lehtonen@gmail.com> 78401 .IP \[bu] 2 78402 Matei David <matei.david@gmail.com> 78403 .IP \[bu] 2 78404 David <david.bramwell@endemolshine.com> 78405 .IP \[bu] 2 78406 Anthony Rusdi <33247310+antrusd@users.noreply.github.com> 78407 .IP \[bu] 2 78408 Richard Patel <me@terorie.dev> 78409 .IP \[bu] 2 78410 \[u5E84]\[u5929]\[u7FFC] <zty0826@gmail.com> 78411 .IP \[bu] 2 78412 SwitchJS <dev@switchjs.com> 78413 .IP \[bu] 2 78414 Raphael <PowershellNinja@users.noreply.github.com> 78415 .IP \[bu] 2 78416 Sezal Agrawal <sezalagrawal@gmail.com> 78417 .IP \[bu] 2 78418 Tyler <TylerNakamura@users.noreply.github.com> 78419 .IP \[bu] 2 78420 Brett Dutro <brett.dutro@gmail.com> 78421 .IP \[bu] 2 78422 Vighnesh SK <booterror99@gmail.com> 78423 .IP \[bu] 2 78424 Arijit Biswas <dibbyo456@gmail.com> 78425 .IP \[bu] 2 78426 Michele Caci <michele.caci@gmail.com> 78427 .IP \[bu] 2 78428 AlexandrBoltris <ua2fgb@gmail.com> 78429 .IP \[bu] 2 78430 Bryce Larson <blarson@saltstack.com> 78431 .IP \[bu] 2 78432 Carlos Ferreyra <crypticmind@gmail.com> 78433 .IP \[bu] 2 78434 Saksham Khanna <sakshamkhanna@outlook.com> 78435 .IP \[bu] 2 78436 dausruddin <5763466+dausruddin@users.noreply.github.com> 78437 .IP \[bu] 2 78438 zero-24 <zero-24@users.noreply.github.com> 78439 .IP \[bu] 2 78440 Xiaoxing Ye <ye@xiaoxing.us> 78441 .IP \[bu] 2 78442 Barry Muldrey <barry@muldrey.net> 78443 .IP \[bu] 2 78444 Sebastian Brandt <sebastian.brandt@friday.de> 78445 .IP \[bu] 2 78446 Marco Molteni <marco.molteni@mailbox.org> 78447 .IP \[bu] 2 78448 Ankur Gupta <7876747+ankur0493@users.noreply.github.com> 78449 .IP \[bu] 2 78450 Maciej Zimnoch <maciej@scylladb.com> 78451 .IP \[bu] 2 78452 anuar45 <serdaliyev.anuar@gmail.com> 78453 .IP \[bu] 2 78454 Fernando <ferferga@users.noreply.github.com> 78455 .IP \[bu] 2 78456 David Cole <david.cole@sohonet.com> 78457 .IP \[bu] 2 78458 Wei He <git@weispot.com> 78459 .IP \[bu] 2 78460 Outvi V <19144373+outloudvi@users.noreply.github.com> 78461 .IP \[bu] 2 78462 Thomas Kriechbaumer <thomas@kriechbaumer.name> 78463 .IP \[bu] 2 78464 Tennix <tennix@users.noreply.github.com> 78465 .IP \[bu] 2 78466 Ole Sch\[:u]tt <ole@schuett.name> 78467 .IP \[bu] 2 78468 Kuang-che Wu <kcwu@csie.org> 78469 .IP \[bu] 2 78470 Thomas Eales <wingsuit@users.noreply.github.com> 78471 .IP \[bu] 2 78472 Paul Tinsley <paul.tinsley@vitalsource.com> 78473 .IP \[bu] 2 78474 Felix Hungenberg <git@shiftgeist.com> 78475 .IP \[bu] 2 78476 Benjamin Richter <github@dev.telepath.de> 78477 .IP \[bu] 2 78478 landall <cst_zf@qq.com> 78479 .IP \[bu] 2 78480 thestigma <thestigma@gmail.com> 78481 .IP \[bu] 2 78482 jtagcat <38327267+jtagcat@users.noreply.github.com> 78483 .IP \[bu] 2 78484 Damon Permezel <permezel@me.com> 78485 .IP \[bu] 2 78486 boosh <boosh@users.noreply.github.com> 78487 .IP \[bu] 2 78488 unbelauscht <58393353+unbelauscht@users.noreply.github.com> 78489 .IP \[bu] 2 78490 Motonori IWAMURO <vmi@nifty.com> 78491 .IP \[bu] 2 78492 Benjapol Worakan <benwrk@live.com> 78493 .IP \[bu] 2 78494 Dave Koston <dave.koston@stackpath.com> 78495 .IP \[bu] 2 78496 Durval Menezes <DurvalMenezes@users.noreply.github.com> 78497 .IP \[bu] 2 78498 Tim Gallant <me@timgallant.us> 78499 .IP \[bu] 2 78500 Frederick Zhang <frederick888@tsundere.moe> 78501 .IP \[bu] 2 78502 valery1707 <valery1707@gmail.com> 78503 .IP \[bu] 2 78504 Yves G <theYinYeti@yalis.fr> 78505 .IP \[bu] 2 78506 Shing Kit Chan <chanshingkit@gmail.com> 78507 .IP \[bu] 2 78508 Franklyn Tackitt <franklyn@tackitt.net> 78509 .IP \[bu] 2 78510 Robert-Andr\['e] Mauchin <zebob.m@gmail.com> 78511 .IP \[bu] 2 78512 evileye <48332831+ibiruai@users.noreply.github.com> 78513 .IP \[bu] 2 78514 Joachim Brandon LeBlanc <brandon@leblanc.codes> 78515 .IP \[bu] 2 78516 Patryk Jakuszew <patryk.jakuszew@gmail.com> 78517 .IP \[bu] 2 78518 fishbullet <shindu666@gmail.com> 78519 .IP \[bu] 2 78520 greatroar <\[at]> 78521 .IP \[bu] 2 78522 Bernd Schoolmann <mail@quexten.com> 78523 .IP \[bu] 2 78524 Elan Ruusam\[:a]e <glen@pld-linux.org> 78525 .IP \[bu] 2 78526 Max Sum <max@lolyculture.com> 78527 .IP \[bu] 2 78528 Mark Spieth <mspieth@users.noreply.github.com> 78529 .IP \[bu] 2 78530 harry <me@harry.plus> 78531 .IP \[bu] 2 78532 Samantha McVey <samantham@posteo.net> 78533 .IP \[bu] 2 78534 Jack Anderson <jack.anderson@metaswitch.com> 78535 .IP \[bu] 2 78536 Michael G <draget@speciesm.net> 78537 .IP \[bu] 2 78538 Brandon Philips <brandon@ifup.org> 78539 .IP \[bu] 2 78540 Daven <dooven@users.noreply.github.com> 78541 .IP \[bu] 2 78542 Martin Stone <martin@d7415.co.uk> 78543 .IP \[bu] 2 78544 David Bramwell <13053834+dbramwell@users.noreply.github.com> 78545 .IP \[bu] 2 78546 Sunil Patra <snl_su@live.com> 78547 .IP \[bu] 2 78548 Adam Stroud <adam.stroud@gmail.com> 78549 .IP \[bu] 2 78550 Kush <kushsharma@users.noreply.github.com> 78551 .IP \[bu] 2 78552 Matan Rosenberg <matan129@gmail.com> 78553 .IP \[bu] 2 78554 gitch1 <63495046+gitch1@users.noreply.github.com> 78555 .IP \[bu] 2 78556 ElonH <elonhhuang@gmail.com> 78557 .IP \[bu] 2 78558 Fred <fred@creativeprojects.tech> 78559 .IP \[bu] 2 78560 S\['e]bastien Gross <renard@users.noreply.github.com> 78561 .IP \[bu] 2 78562 Maxime Suret <11944422+msuret@users.noreply.github.com> 78563 .IP \[bu] 2 78564 Caleb Case <caleb@storj.io> <calebcase@gmail.com> 78565 .IP \[bu] 2 78566 Ben Zenker <imbenzenker@gmail.com> 78567 .IP \[bu] 2 78568 Martin Michlmayr <tbm@cyrius.com> 78569 .IP \[bu] 2 78570 Brandon McNama <bmcnama@pagerduty.com> 78571 .IP \[bu] 2 78572 Daniel Slyman <github@skylayer.eu> 78573 .IP \[bu] 2 78574 Alex Guerrero <guerrero@users.noreply.github.com> 78575 .IP \[bu] 2 78576 Matteo Pietro Dazzi <matteopietro.dazzi@gft.com> 78577 .IP \[bu] 2 78578 edwardxml <56691903+edwardxml@users.noreply.github.com> 78579 .IP \[bu] 2 78580 Roman Kredentser <shareed2k@gmail.com> 78581 .IP \[bu] 2 78582 Kamil Trzci\[u0144]ski <ayufan@ayufan.eu> 78583 .IP \[bu] 2 78584 Zac Rubin <z-0@users.noreply.github.com> 78585 .IP \[bu] 2 78586 Vincent Feltz <psycho@feltzv.fr> 78587 .IP \[bu] 2 78588 Heiko Bornholdt <bornholdt@informatik.uni-hamburg.de> 78589 .IP \[bu] 2 78590 Matteo Pietro Dazzi <matteopietro.dazzi@gmail.com> 78591 .IP \[bu] 2 78592 jtagcat <gitlab@c7.ee> 78593 .IP \[bu] 2 78594 Petri Salminen <petri@salminen.dev> 78595 .IP \[bu] 2 78596 Tim Burke <tim.burke@gmail.com> 78597 .IP \[bu] 2 78598 Kai L\[:u]ke <kai@kinvolk.io> 78599 .IP \[bu] 2 78600 Garrett Squire <github@garrettsquire.com> 78601 .IP \[bu] 2 78602 Evan Harris <eharris@puremagic.com> 78603 .IP \[bu] 2 78604 Kevin <keyam@microsoft.com> 78605 .IP \[bu] 2 78606 Morten Linderud <morten@linderud.pw> 78607 .IP \[bu] 2 78608 Dmitry Ustalov <dmitry.ustalov@gmail.com> 78609 .IP \[bu] 2 78610 Jack <196648+jdeng@users.noreply.github.com> 78611 .IP \[bu] 2 78612 kcris <cristian.tarsoaga@gmail.com> 78613 .IP \[bu] 2 78614 tyhuber1 <68970760+tyhuber1@users.noreply.github.com> 78615 .IP \[bu] 2 78616 David Ibarra <david.ibarra@realty.com> 78617 .IP \[bu] 2 78618 Tim Gallant <tim@lilt.com> 78619 .IP \[bu] 2 78620 Kaloyan Raev <kaloyan@storj.io> 78621 .IP \[bu] 2 78622 Jay McEntire <jay.mcentire@gmail.com> 78623 .IP \[bu] 2 78624 Leo Luan <leoluan@us.ibm.com> 78625 .IP \[bu] 2 78626 aus <549081+aus@users.noreply.github.com> 78627 .IP \[bu] 2 78628 Aaron Gokaslan <agokaslan@fb.com> 78629 .IP \[bu] 2 78630 Egor Margineanu <egmar@users.noreply.github.com> 78631 .IP \[bu] 2 78632 Lucas Kanashiro <lucas.kanashiro@canonical.com> 78633 .IP \[bu] 2 78634 WarpedPixel <WarpedPixel@users.noreply.github.com> 78635 .IP \[bu] 2 78636 Sam Edwards <sam@samedwards.ca> 78637 .IP \[bu] 2 78638 wjielai <gouki0123@gmail.com> 78639 .IP \[bu] 2 78640 Muffin King <jinxz_k@live.com> 78641 .IP \[bu] 2 78642 Christopher Stewart <6573710+1f47a@users.noreply.github.com> 78643 .IP \[bu] 2 78644 Russell Cattelan <cattelan@digitalelves.com> 78645 .IP \[bu] 2 78646 gyutw <30371241+gyutw@users.noreply.github.com> 78647 .IP \[bu] 2 78648 Hekmon <edouardhur@gmail.com> 78649 .IP \[bu] 2 78650 LaSombra <lasombra@users.noreply.github.com> 78651 .IP \[bu] 2 78652 Dov Murik <dov.murik@gmail.com> 78653 .IP \[bu] 2 78654 Ameer Dawood <ameer1234567890@gmail.com> 78655 .IP \[bu] 2 78656 Dan Hipschman <dan.hipschman@opendoor.com> 78657 .IP \[bu] 2 78658 Josh Soref <jsoref@users.noreply.github.com> 78659 .IP \[bu] 2 78660 David <david@staron.nl> 78661 .IP \[bu] 2 78662 Ingo <ingo@hoffmann.cx> 78663 .IP \[bu] 2 78664 Adam Pl\['a]nsk\['y] <adamplansky@users.noreply.github.com> 78665 <adamplansky@gmail.com> 78666 .IP \[bu] 2 78667 Manish Gupta <manishgupta.ait@gmail.com> 78668 .IP \[bu] 2 78669 Deepak Sah <sah.sslpu@gmail.com> 78670 .IP \[bu] 2 78671 Marcin Zelent <marcin@zelent.net> 78672 .IP \[bu] 2 78673 zhucan <zhucan.k8s@gmail.com> 78674 .IP \[bu] 2 78675 James Lim <james.lim@samsara.com> 78676 .IP \[bu] 2 78677 Laurens Janssen <BD69BM@insim.biz> 78678 .IP \[bu] 2 78679 Bob Bagwill <bobbagwill@gmail.com> 78680 .IP \[bu] 2 78681 Nathan Collins <colli372@msu.edu> 78682 .IP \[bu] 2 78683 lostheli 78684 .IP \[bu] 2 78685 kelv <kelvin@acks.org> 78686 .IP \[bu] 2 78687 Milly <milly.ca@gmail.com> 78688 .IP \[bu] 2 78689 gtorelly <gtorelly@gmail.com> 78690 .IP \[bu] 2 78691 Brad Ackerman <brad@facefault.org> 78692 .IP \[bu] 2 78693 Mitsuo Heijo <mitsuo.heijo@gmail.com> 78694 .IP \[bu] 2 78695 Claudio Bantaloukas <rockdreamer@gmail.com> 78696 .IP \[bu] 2 78697 Benjamin Gustin <gustin.ben@gmail.com> 78698 .IP \[bu] 2 78699 Ingo Weiss <ingo@redhat.com> 78700 .IP \[bu] 2 78701 Kerry Su <me@sshockwave.net> 78702 .IP \[bu] 2 78703 Ilyess Bachiri <ilyess.bachiri@sonder.com> 78704 .IP \[bu] 2 78705 Yury Stankevich <urykhy@gmail.com> 78706 .IP \[bu] 2 78707 kice <wslikerqs@gmail.com> 78708 .IP \[bu] 2 78709 Denis Neuling <denisneuling@gmail.com> 78710 .IP \[bu] 2 78711 Janne Johansson <icepic.dz@gmail.com> 78712 .IP \[bu] 2 78713 Patrik Nordl\['e]n <patriki@gmail.com> 78714 .IP \[bu] 2 78715 CokeMine <aptx4561@gmail.com> 78716 .IP \[bu] 2 78717 S\[u01A1]n Tr\[u1EA7]n-Nguy\[u1EC5]n <github@sntran.com> 78718 .IP \[bu] 2 78719 lluuaapp <266615+lluuaapp@users.noreply.github.com> 78720 .IP \[bu] 2 78721 Zach Kipp <kipp.zach@gmail.com> 78722 .IP \[bu] 2 78723 Riccardo Iaconelli <riccardo@kde.org> 78724 .IP \[bu] 2 78725 Sakuragawa Misty <gyc990326@gmail.com> 78726 .IP \[bu] 2 78727 Nicolas Rueff <nicolas@rueff.fr> 78728 .IP \[bu] 2 78729 Pau Rodriguez-Estivill <prodrigestivill@gmail.com> 78730 .IP \[bu] 2 78731 Bob Pusateri <BobPusateri@users.noreply.github.com> 78732 .IP \[bu] 2 78733 Alex JOST <25005220+dimejo@users.noreply.github.com> 78734 .IP \[bu] 2 78735 Alexey Tabakman <samosad.ru@gmail.com> 78736 .IP \[bu] 2 78737 David Sze <sze.david@gmail.com> 78738 .IP \[bu] 2 78739 cynthia kwok <cynthia.m.kwok@gmail.com> 78740 .IP \[bu] 2 78741 Miron Veryanskiy <MironVeryanskiy@gmail.com> 78742 .IP \[bu] 2 78743 K265 <k.265@qq.com> 78744 .IP \[bu] 2 78745 Vesnyx <Vesnyx@users.noreply.github.com> 78746 .IP \[bu] 2 78747 Dmitry Chepurovskiy <me@dm3ch.net> 78748 .IP \[bu] 2 78749 Rauno Ots <rauno.ots@cgi.com> 78750 .IP \[bu] 2 78751 Georg Neugschwandtner <georg.neugschwandtner@gmx.net> 78752 .IP \[bu] 2 78753 pvalls <polvallsrue@gmail.com> 78754 .IP \[bu] 2 78755 Robert Thomas <31854736+wolveix@users.noreply.github.com> 78756 .IP \[bu] 2 78757 Romeo Kienzler <romeo.kienzler@gmail.com> 78758 .IP \[bu] 2 78759 tYYGH <tYYGH@users.noreply.github.com> 78760 .IP \[bu] 2 78761 georne <77802995+georne@users.noreply.github.com> 78762 .IP \[bu] 2 78763 Maxwell Calman <mcalman@MacBook-Pro.local> 78764 .IP \[bu] 2 78765 Naveen Honest Raj <naveendurai19@gmail.com> 78766 .IP \[bu] 2 78767 Lucas Messenger <lmesseng@cisco.com> 78768 .IP \[bu] 2 78769 Manish Kumar <krmanish260@gmail.com> 78770 .IP \[bu] 2 78771 x0b <x0bdev@gmail.com> 78772 .IP \[bu] 2 78773 CERN through the CS3MESH4EOSC Project 78774 .IP \[bu] 2 78775 Nick Gaya <nicholasgaya+github@gmail.com> 78776 .IP \[bu] 2 78777 Ashok Gelal <401055+ashokgelal@users.noreply.github.com> 78778 .IP \[bu] 2 78779 Dominik Mydlil <dominik.mydlil@outlook.com> 78780 .IP \[bu] 2 78781 Nazar Mishturak <nazarmx@gmail.com> 78782 .IP \[bu] 2 78783 Ansh Mittal <iamAnshMittal@gmail.com> 78784 .IP \[bu] 2 78785 noabody <noabody@yahoo.com> 78786 .IP \[bu] 2 78787 OleFrost <82263101+olefrost@users.noreply.github.com> 78788 .IP \[bu] 2 78789 Kenny Parsons <kennyparsons93@gmail.com> 78790 .IP \[bu] 2 78791 Jeffrey Tolar <tolar.jeffrey@gmail.com> 78792 .IP \[bu] 2 78793 jtagcat <git-514635f7@jtag.cat> 78794 .IP \[bu] 2 78795 Tatsuya Noyori <63089076+public-tatsuya-noyori@users.noreply.github.com> 78796 .IP \[bu] 2 78797 lewisxy <lewisxy@users.noreply.github.com> 78798 .IP \[bu] 2 78799 Nolan Woods <nolan_w@sfu.ca> 78800 .IP \[bu] 2 78801 Gautam Kumar <25435568+gautamajay52@users.noreply.github.com> 78802 .IP \[bu] 2 78803 Chris Macklin <chris.macklin@10xgenomics.com> 78804 .IP \[bu] 2 78805 Antoon Prins <antoon.prins@surfsara.nl> 78806 .IP \[bu] 2 78807 Alexey Ivanov <rbtz@dropbox.com> 78808 .IP \[bu] 2 78809 Serge Pouliquen <sp31415@free.fr> 78810 .IP \[bu] 2 78811 acsfer <carlos@reendex.com> 78812 .IP \[bu] 2 78813 Tom <tom@tom-fitzhenry.me.uk> 78814 .IP \[bu] 2 78815 Tyson Moore <tyson@tyson.me> 78816 .IP \[bu] 2 78817 database64128 <free122448@hotmail.com> 78818 .IP \[bu] 2 78819 Chris Lu <chrislusf@users.noreply.github.com> 78820 .IP \[bu] 2 78821 Reid Buzby <reid@rethink.software> 78822 .IP \[bu] 2 78823 darrenrhs <darrenrhs@gmail.com> 78824 .IP \[bu] 2 78825 Florian Penzkofer <fp@nullptr.de> 78826 .IP \[bu] 2 78827 Xuanchen Wu <117010292@link.cuhk.edu.cn> 78828 .IP \[bu] 2 78829 partev <petrosyan@gmail.com> 78830 .IP \[bu] 2 78831 Dmitry Sitnikov <fo2@inbox.ru> 78832 .IP \[bu] 2 78833 Haochen Tong <i@hexchain.org> 78834 .IP \[bu] 2 78835 Michael Hanselmann <public@hansmi.ch> 78836 .IP \[bu] 2 78837 Chuan Zh <zhchuan7@gmail.com> 78838 .IP \[bu] 2 78839 Antoine GIRARD <antoine.girard@sapk.fr> 78840 .IP \[bu] 2 78841 Justin Winokur (Jwink3101) <Jwink3101@users.noreply.github.com> 78842 .IP \[bu] 2 78843 Mariano Absatz (git) <scm@baby.com.ar> 78844 .IP \[bu] 2 78845 Greg Sadetsky <lepetitg@gmail.com> 78846 .IP \[bu] 2 78847 yedamo <logindaveye@gmail.com> 78848 .IP \[bu] 2 78849 hota <lindwurm.q@gmail.com> 78850 .IP \[bu] 2 78851 vinibali <vinibali1@gmail.com> 78852 .IP \[bu] 2 78853 Ken Enrique Morel <ken.morel.santana@gmail.com> 78854 .IP \[bu] 2 78855 Justin Hellings <justin.hellings@gmail.com> 78856 .IP \[bu] 2 78857 Parth Shukla <pparth@pparth.net> 78858 .IP \[bu] 2 78859 wzl <wangzl31@outlook.com> 78860 .IP \[bu] 2 78861 HNGamingUK <connor@earnshawhome.co.uk> 78862 .IP \[bu] 2 78863 Jonta <359397+Jonta@users.noreply.github.com> 78864 .IP \[bu] 2 78865 YenForYang <YenForYang@users.noreply.github.com> 78866 .IP \[bu] 2 78867 SimJoSt / Joda St\[:o]\[ss]er <git@simjo.st> 78868 .IP \[bu] 2 78869 Logeshwaran <waranlogesh@gmail.com> 78870 .IP \[bu] 2 78871 Rajat Goel <rajat@dropbox.com> 78872 .IP \[bu] 2 78873 r0kk3rz <r0kk3rz@gmail.com> 78874 .IP \[bu] 2 78875 Matthew Sevey <mjsevey@gmail.com> 78876 .IP \[bu] 2 78877 Filip Rysavy <fil@siasky.net> 78878 .IP \[bu] 2 78879 Ian Levesque <ian@ianlevesque.org> 78880 .IP \[bu] 2 78881 Thomas Stachl <thomas@stachl.me> 78882 .IP \[bu] 2 78883 Dmitry Bogatov <git#v1@kaction.cc> 78884 .IP \[bu] 2 78885 thomae <4493560+thomae@users.noreply.github.com> 78886 .IP \[bu] 2 78887 trevyn <trevyn-git@protonmail.com> 78888 .IP \[bu] 2 78889 David Liu <david.yx.liu@oracle.com> 78890 .IP \[bu] 2 78891 Chris Nelson <stuff@cjnaz.com> 78892 .IP \[bu] 2 78893 Felix Bu\[u0308]nemann <felix.buenemann@gmail.com> 78894 .IP \[bu] 2 78895 At\['i]lio Ant\[^o]nio <atiliodadalto@hotmail.com> 78896 .IP \[bu] 2 78897 Carlo Mion <mion00@gmail.com> 78898 .IP \[bu] 2 78899 Chris Lu <chris.lu@gmail.com> 78900 .IP \[bu] 2 78901 Vitor Arruda <vitor.pimenta.arruda@gmail.com> 78902 .IP \[bu] 2 78903 bbabich <bbabich@datamossa.com> 78904 .IP \[bu] 2 78905 David <dp.davide.palma@gmail.com> 78906 .IP \[bu] 2 78907 Borna Butkovic <borna@favicode.net> 78908 .IP \[bu] 2 78909 Fredric Arklid <fredric.arklid@consid.se> 78910 .IP \[bu] 2 78911 Andy Jackson <Andrew.Jackson@bl.uk> 78912 .IP \[bu] 2 78913 Sinan Tan <i@tinytangent.com> 78914 .IP \[bu] 2 78915 deinferno <14363193+deinferno@users.noreply.github.com> 78916 .IP \[bu] 2 78917 rsapkf <rsapkfff@pm.me> 78918 .IP \[bu] 2 78919 Will Holtz <wholtz@gmail.com> 78920 .IP \[bu] 2 78921 GGG KILLER <gggkiller2@gmail.com> 78922 .IP \[bu] 2 78923 Logeshwaran Murugesan <logeshwaran@testpress.in> 78924 .IP \[bu] 2 78925 Lu Wang <coolwanglu@gmail.com> 78926 .IP \[bu] 2 78927 Bumsu Hyeon <ksitht@gmail.com> 78928 .IP \[bu] 2 78929 Shmz Ozggrn <98463324+ShmzOzggrn@users.noreply.github.com> 78930 .IP \[bu] 2 78931 Kim <kim@jotta.no> 78932 .IP \[bu] 2 78933 Niels van de Weem <n.van.de.weem@smile.nl> 78934 .IP \[bu] 2 78935 Koopa <codingkoopa@gmail.com> 78936 .IP \[bu] 2 78937 Yunhai Luo <yunhai-luo@hotmail.com> 78938 .IP \[bu] 2 78939 Charlie Jiang <w@chariri.moe> 78940 .IP \[bu] 2 78941 Alain Nussbaumer <alain.nussbaumer@alleluia.ch> 78942 .IP \[bu] 2 78943 Vanessasaurus <814322+vsoch@users.noreply.github.com> 78944 .IP \[bu] 2 78945 Isaac Levy <isaac.r.levy@gmail.com> 78946 .IP \[bu] 2 78947 Gourav T <workflowautomation@protonmail.com> 78948 .IP \[bu] 2 78949 Paulo Martins <paulo.pontes.m@gmail.com> 78950 .IP \[bu] 2 78951 viveknathani <viveknathani2402@gmail.com> 78952 .IP \[bu] 2 78953 Eng Zer Jun <engzerjun@gmail.com> 78954 .IP \[bu] 2 78955 Abhiraj <abhiraj.official15@gmail.com> 78956 .IP \[bu] 2 78957 M\['a]rton Elek <elek@apache.org> <elek@users.noreply.github.com> 78958 .IP \[bu] 2 78959 Vincent Murphy <vdm@vdm.ie> 78960 .IP \[bu] 2 78961 ctrl-q <34975747+ctrl-q@users.noreply.github.com> 78962 .IP \[bu] 2 78963 Nil Alexandrov <nalexand@akamai.com> 78964 .IP \[bu] 2 78965 GuoXingbin <101376330+guoxingbin@users.noreply.github.com> 78966 .IP \[bu] 2 78967 Berkan Teber <berkan@berkanteber.com> 78968 .IP \[bu] 2 78969 Tobias Klauser <tklauser@distanz.ch> 78970 .IP \[bu] 2 78971 KARBOWSKI Piotr <piotr.karbowski@gmail.com> 78972 .IP \[bu] 2 78973 GH <geeklihui@foxmail.com> 78974 .IP \[bu] 2 78975 rafma0 <int.main@gmail.com> 78976 .IP \[bu] 2 78977 Adrien Rey-Jarthon <jobs@adrienjarthon.com> 78978 .IP \[bu] 2 78979 Nick Gooding <73336146+nickgooding@users.noreply.github.com> 78980 .IP \[bu] 2 78981 Leroy van Logchem <lr.vanlogchem@gmail.com> 78982 .IP \[bu] 2 78983 Zsolt Ero <zsolt.ero@gmail.com> 78984 .IP \[bu] 2 78985 Lesmiscore <nao20010128@gmail.com> 78986 .IP \[bu] 2 78987 ehsantdy <ehsan.tadayon@arvancloud.com> <ehsantadayon85@gmail.com> 78988 .IP \[bu] 2 78989 SwazRGB <65694696+swazrgb@users.noreply.github.com> 78990 .IP \[bu] 2 78991 Mateusz Puczyn\[u0301]ski <mati6095@gmail.com> 78992 .IP \[bu] 2 78993 Michael C Tiernan - MIT-Research Computing Project <mtiernan@mit.edu> 78994 .IP \[bu] 2 78995 Kaspian <34658474+KaspianDev@users.noreply.github.com> 78996 .IP \[bu] 2 78997 Werner <EvilOlaf@users.noreply.github.com> 78998 .IP \[bu] 2 78999 Hugal31 <hugo.laloge@gmail.com> 79000 .IP \[bu] 2 79001 Christian Galo <36752715+cgalo5758@users.noreply.github.com> 79002 .IP \[bu] 2 79003 Erik van Velzen <erik@evanv.nl> 79004 .IP \[bu] 2 79005 Derek Battams <derek@battams.ca> 79006 .IP \[bu] 2 79007 Paul <devnoname120@gmail.com> 79008 .IP \[bu] 2 79009 SimonLiu <simonliu009@users.noreply.github.com> 79010 .IP \[bu] 2 79011 Hugo Laloge <hla@lescompanions.com> 79012 .IP \[bu] 2 79013 Mr-Kanister <68117355+Mr-Kanister@users.noreply.github.com> 79014 .IP \[bu] 2 79015 Rob Pickerill <r.pickerill@gmail.com> 79016 .IP \[bu] 2 79017 Andrey <to.merge@gmail.com> 79018 .IP \[bu] 2 79019 Eric Wolf <19wolf@gmail.com> 79020 .IP \[bu] 2 79021 Nick <nick.naumann@mailbox.tu-dresden.de> 79022 .IP \[bu] 2 79023 Jason Zheng <jszheng17@gmail.com> 79024 .IP \[bu] 2 79025 Matthew Vernon <mvernon@wikimedia.org> 79026 .IP \[bu] 2 79027 Noah Hsu <i@nn.ci> 79028 .IP \[bu] 2 79029 m00594701 <mengpengbo@huawei.com> 79030 .IP \[bu] 2 79031 Art M. 79032 Gallagher <artmg50@gmail.com> 79033 .IP \[bu] 2 79034 Sven Gerber <49589423+svengerber@users.noreply.github.com> 79035 .IP \[bu] 2 79036 CrossR <r.cross@lancaster.ac.uk> 79037 .IP \[bu] 2 79038 Maciej Radzikowski <maciej@radzikowski.com.pl> 79039 .IP \[bu] 2 79040 Scott Grimes <scott.grimes@spaciq.com> 79041 .IP \[bu] 2 79042 Phil Shackleton <71221528+philshacks@users.noreply.github.com> 79043 .IP \[bu] 2 79044 eNV25 <env252525@gmail.com> 79045 .IP \[bu] 2 79046 Caleb <inventor96@users.noreply.github.com> 79047 .IP \[bu] 2 79048 J-P Treen <jp@wraptious.com> 79049 .IP \[bu] 2 79050 Martin Czygan <53705+miku@users.noreply.github.com> 79051 .IP \[bu] 2 79052 buda <sandrojijavadze@protonmail.com> 79053 .IP \[bu] 2 79054 mirekphd <36706320+mirekphd@users.noreply.github.com> 79055 .IP \[bu] 2 79056 vyloy <vyloy@qq.com> 79057 .IP \[bu] 2 79058 Anthrazz <25553648+Anthrazz@users.noreply.github.com> 79059 .IP \[bu] 2 79060 zzr93 <34027824+zzr93@users.noreply.github.com> 79061 .IP \[bu] 2 79062 Paul Norman <penorman@mac.com> 79063 .IP \[bu] 2 79064 Lorenzo Maiorfi <maiorfi@gmail.com> 79065 .IP \[bu] 2 79066 Claudio Maradonna <penguyman@stronzi.org> 79067 .IP \[bu] 2 79068 Ovidiu Victor Tatar <ovi.tatar@googlemail.com> 79069 .IP \[bu] 2 79070 Evan Spensley <epspensley@gmail.com> 79071 .IP \[bu] 2 79072 Yen Hu <61753151+0x59656e@users.noreply.github.com> 79073 .IP \[bu] 2 79074 Steve Kowalik <steven@wedontsleep.org> 79075 .IP \[bu] 2 79076 Jordi Gonzalez Mu\[~n]oz <jordigonzm@gmail.com> 79077 .IP \[bu] 2 79078 Joram Schrijver <i@joram.io> 79079 .IP \[bu] 2 79080 Mark Trolley <marktrolley@gmail.com> 79081 .IP \[bu] 2 79082 Jo\[~a]o Henrique Franco <joaohenrique.franco@gmail.com> 79083 .IP \[bu] 2 79084 anonion <aman207@users.noreply.github.com> 79085 .IP \[bu] 2 79086 Ryan Morey <4590343+rmorey@users.noreply.github.com> 79087 .IP \[bu] 2 79088 Simon Bos <simonbos9@gmail.com> 79089 .IP \[bu] 2 79090 YFdyh000 <yfdyh000@gmail.com> * Josh Soref 79091 <2119212+jsoref@users.noreply.github.com> 79092 .IP \[bu] 2 79093 \[/O]yvind Heddeland Instefjord <instefjord@outlook.com> 79094 .IP \[bu] 2 79095 Dmitry Deniskin <110819396+ddeniskin@users.noreply.github.com> 79096 .IP \[bu] 2 79097 Alexander Knorr <106825+opexxx@users.noreply.github.com> 79098 .IP \[bu] 2 79099 Richard Bateman <richard@batemansr.us> 79100 .IP \[bu] 2 79101 Dimitri Papadopoulos Orfanos 79102 <3234522+DimitriPapadopoulos@users.noreply.github.com> 79103 .IP \[bu] 2 79104 Lorenzo Milesi <lorenzo.milesi@yetopen.com> 79105 .IP \[bu] 2 79106 Isaac Aymerich <isaac.aymerich@gmail.com> 79107 .IP \[bu] 2 79108 YanceyChiew <35898533+YanceyChiew@users.noreply.github.com> 79109 .IP \[bu] 2 79110 Manoj Ghosh <msays2000@gmail.com> 79111 .IP \[bu] 2 79112 Bachue Zhou <bachue.shu@gmail.com> 79113 .IP \[bu] 2 79114 Manoj Ghosh <manoj.ghosh@oracle.com> 79115 .IP \[bu] 2 79116 Tom Mombourquette <tom@devnode.com> 79117 .IP \[bu] 2 79118 Robert Newson <rnewson@apache.org> 79119 .IP \[bu] 2 79120 Samuel Johnson <esamueljohnson@gmail.com> 79121 .IP \[bu] 2 79122 coultonluke <luke@luke.org.uk> 79123 .IP \[bu] 2 79124 Anthony Pessy <anthony@cogniteev.com> 79125 .IP \[bu] 2 79126 Philip Harvey <pharvey@battelleecology.org> 79127 .IP \[bu] 2 79128 dgouju <dgouju@users.noreply.github.com> 79129 .IP \[bu] 2 79130 Cl\['e]ment Notin <clement.notin@gmail.com> 79131 .IP \[bu] 2 79132 x3-apptech <66947598+x3-apptech@users.noreply.github.com> 79133 .IP \[bu] 2 79134 Arnie97 <arnie97@gmail.com> 79135 .IP \[bu] 2 79136 Roel Arents <2691308+roelarents@users.noreply.github.com> 79137 .IP \[bu] 2 79138 Aaron Gokaslan <aaronGokaslan@gmail.com> 79139 .IP \[bu] 2 79140 techknowlogick <matti@mdranta.net> 79141 .IP \[bu] 2 79142 rkettelerij <richard@mindloops.nl> 79143 .IP \[bu] 2 79144 Kamui <fin-kamui@pm.me> 79145 .IP \[bu] 2 79146 asdffdsazqqq <90116442+asdffdsazqqq@users.noreply.github.com> 79147 .IP \[bu] 2 79148 Nathaniel Wesley Filardo <nfilardo@microsoft.com> 79149 .IP \[bu] 2 79150 ycdtosa <ycdtosa@users.noreply.github.com> 79151 .IP \[bu] 2 79152 Erik Agterdenbos <agterdenbos@users.noreply.github.com> 79153 .IP \[bu] 2 79154 Kevin Verstaen <48050031+kverstae@users.noreply.github.com> 79155 .IP \[bu] 2 79156 MohammadReza <mrvashian@gmail.com> 79157 .IP \[bu] 2 79158 vanplus <60313789+vanplus@users.noreply.github.com> 79159 .IP \[bu] 2 79160 Jack <16779171+jkpe@users.noreply.github.com> 79161 .IP \[bu] 2 79162 Abdullah Saglam <abdullah.saglam@stonebranch.com> 79163 .IP \[bu] 2 79164 Marks Polakovs <github@markspolakovs.me> 79165 .IP \[bu] 2 79166 piyushgarg <piyushgarg80@gmail.com> 79167 .IP \[bu] 2 79168 Kaloyan Raev <kaloyan-raev@users.noreply.github.com> 79169 .IP \[bu] 2 79170 IMTheNachoMan <imthenachoman@gmail.com> 79171 .IP \[bu] 2 79172 alankrit <alankrit@google.com> 79173 .IP \[bu] 2 79174 Bryan Kaplan <#\[at]bryankaplan.com> 79175 .IP \[bu] 2 79176 LXY <767763591@qq.com> 79177 .IP \[bu] 2 79178 Simmon Li (he/him) <li.simmon@gmail.com> 79179 .IP \[bu] 2 79180 happyxhw <44490504+happyxhw@users.noreply.github.com> 79181 .IP \[bu] 2 79182 Simmon Li (he/him) <hello@crespire.dev> 79183 .IP \[bu] 2 79184 Matthias Baur <baurmatt@users.noreply.github.com> 79185 .IP \[bu] 2 79186 Hunter Wittenborn <hunter@hunterwittenborn.com> 79187 .IP \[bu] 2 79188 logopk <peter@kreuser.name> 79189 .IP \[bu] 2 79190 Gerard Bosch <30733556+gerardbosch@users.noreply.github.com> 79191 .IP \[bu] 2 79192 ToBeFree <github@tfrei.de> 79193 .IP \[bu] 2 79194 NodudeWasTaken <75137537+NodudeWasTaken@users.noreply.github.com> 79195 .IP \[bu] 2 79196 Peter Brunner <peter@lugoues.net> 79197 .IP \[bu] 2 79198 Ninh Pham <dongian.rapclubkhtn@gmail.com> 79199 .IP \[bu] 2 79200 Ryan Caezar Itang <sitiom@proton.me> 79201 .IP \[bu] 2 79202 Peter Brunner <peter@psykhe.com> 79203 .IP \[bu] 2 79204 Leandro Sacchet <leandro.sacchet@animati.com.br> 79205 .IP \[bu] 2 79206 dependabot[bot] <49699333+dependabot[bot]\[at]users.noreply.github.com> 79207 .IP \[bu] 2 79208 cycneuramus <56681631+cycneuramus@users.noreply.github.com> 79209 .IP \[bu] 2 79210 Arnavion <me@arnavion.dev> 79211 .IP \[bu] 2 79212 Christopher Merry <christopher.merry@mlb.com> 79213 .IP \[bu] 2 79214 Thibault Coupin <thibault.coupin@gmail.com> 79215 .IP \[bu] 2 79216 Richard Tweed <RichardoC@users.noreply.github.com> 79217 .IP \[bu] 2 79218 Zach Kipp <Zacho2@users.noreply.github.com> 79219 .IP \[bu] 2 79220 yuudi <26199752+yuudi@users.noreply.github.com> 79221 .IP \[bu] 2 79222 NickIAm <NickIAm@users.noreply.github.com> 79223 .IP \[bu] 2 79224 Juang, Yi-Lin <frankyjuang@gmail.com> 79225 .IP \[bu] 2 79226 jumbi77 <jumbi77@users.noreply.github.com> 79227 .IP \[bu] 2 79228 Aditya Basu <ab.aditya.basu@gmail.com> 79229 .IP \[bu] 2 79230 ed <s@ocv.me> 79231 .IP \[bu] 2 79232 Drew Parsons <dparsons@emerall.com> 79233 .IP \[bu] 2 79234 Joel <joelnb@users.noreply.github.com> 79235 .IP \[bu] 2 79236 wiserain <mail275@gmail.com> 79237 .IP \[bu] 2 79238 Roel Arents <roel.arents@kadaster.nl> 79239 .IP \[bu] 2 79240 Shyim <github@shyim.de> 79241 .IP \[bu] 2 79242 Rintze Zelle <78232505+rzelle-lallemand@users.noreply.github.com> 79243 .IP \[bu] 2 79244 Damo <damoclark@users.noreply.github.com> 79245 .IP \[bu] 2 79246 WeidiDeng <weidi_deng@icloud.com> 79247 .IP \[bu] 2 79248 Brian Starkey <stark3y@gmail.com> 79249 .IP \[bu] 2 79250 jladbrook <jhladbrook@gmail.com> 79251 .IP \[bu] 2 79252 Loren Gordon <lorengordon@users.noreply.github.com> 79253 .IP \[bu] 2 79254 dlitster <davidlitster@gmail.com> 79255 .IP \[bu] 2 79256 Tobias Gion <tobias@gion.io> 79257 .IP \[bu] 2 79258 J\[u0101]nis Bebr\[u012B]tis <janis.bebritis@wunder.io> 79259 .IP \[bu] 2 79260 Adam K <github.com@ak.tidy.email> 79261 .IP \[bu] 2 79262 Andrei Smirnov <smirnov.captain@gmail.com> 79263 .IP \[bu] 2 79264 Janne Hellsten <jjhellst@gmail.com> 79265 .IP \[bu] 2 79266 cc <12904584+shvc@users.noreply.github.com> 79267 .IP \[bu] 2 79268 Tareq Sharafy <tareq.sha@gmail.com> 79269 .IP \[bu] 2 79270 kapitainsky <dariuszb@me.com> 79271 .IP \[bu] 2 79272 douchen <playgoobug@gmail.com> 79273 .IP \[bu] 2 79274 Sam Lai <70988+slai@users.noreply.github.com> 79275 .IP \[bu] 2 79276 URenko <18209292+URenko@users.noreply.github.com> 79277 .IP \[bu] 2 79278 Stanislav Gromov <kullfar@gmail.com> 79279 .IP \[bu] 2 79280 Paulo Schreiner <paulo.schreiner@delivion.de> 79281 .IP \[bu] 2 79282 Mariusz Suchodolski <mariusz@suchodol.ski> 79283 .IP \[bu] 2 79284 danielkrajnik <dan94kra@gmail.com> 79285 .IP \[bu] 2 79286 Peter Fern <github@0xc0dedbad.com> 79287 .IP \[bu] 2 79288 zzq <i@zhangzqs.cn> 79289 .IP \[bu] 2 79290 mac-15 <usman.ilamdin@phpstudios.com> 79291 .IP \[bu] 2 79292 Sawada Tsunayoshi <34431649+TsunayoshiSawada@users.noreply.github.com> 79293 .IP \[bu] 2 79294 Dean Attali <daattali@gmail.com> 79295 .IP \[bu] 2 79296 Fjodor42 <molgaard@gmail.com> 79297 .IP \[bu] 2 79298 BakaWang <wa11579@hotmail.com> 79299 .IP \[bu] 2 79300 Mahad <56235065+Mahad-lab@users.noreply.github.com> 79301 .IP \[bu] 2 79302 Vladislav Vorobev <x.miere@gmail.com> 79303 .IP \[bu] 2 79304 darix <darix@users.noreply.github.com> 79305 .IP \[bu] 2 79306 Benjamin <36415086+bbenjamin-sys@users.noreply.github.com> 79307 .IP \[bu] 2 79308 Chun-Hung Tseng <henrybear327@users.noreply.github.com> 79309 .IP \[bu] 2 79310 Ricardo D\[aq]O. 79311 Albanus <rdalbanus@users.noreply.github.com> 79312 .IP \[bu] 2 79313 gabriel-suela <gscsuela@gmail.com> 79314 .IP \[bu] 2 79315 Tiago Boeing <contato@tiagoboeing.com> 79316 .IP \[bu] 2 79317 Edwin Mackenzie-Owen <edwin.mowen@gmail.com> 79318 .IP \[bu] 2 79319 Niklas Hamb\[:u]chen <mail@nh2.me> 79320 .IP \[bu] 2 79321 yuudi <yuudi@users.noreply.github.com> 79322 .IP \[bu] 2 79323 Zach <github@prozach.org> 79324 .IP \[bu] 2 79325 nielash <31582349+nielash@users.noreply.github.com> 79326 .IP \[bu] 2 79327 Julian Lepinski <lepinsk@users.noreply.github.com> 79328 .IP \[bu] 2 79329 Raymond Berger <RayBB@users.noreply.github.com> 79330 .IP \[bu] 2 79331 Nihaal Sangha <nihaal.git@gmail.com> 79332 .IP \[bu] 2 79333 Masamune3210 <1053504+Masamune3210@users.noreply.github.com> 79334 .IP \[bu] 2 79335 James Braza <jamesbraza@gmail.com> 79336 .IP \[bu] 2 79337 antoinetran <antoinetran@users.noreply.github.com> 79338 .IP \[bu] 2 79339 alexia <me@alexia.lol> 79340 .IP \[bu] 2 79341 nielash <nielronash@gmail.com> 79342 .IP \[bu] 2 79343 Vitor Gomes <vitor.gomes@delivion.de> <mail@vitorgomes.com> 79344 .IP \[bu] 2 79345 Jacob Hands <jacob@gogit.io> 79346 .IP \[bu] 2 79347 hideo aoyama <100831251+boukendesho@users.noreply.github.com> 79348 .IP \[bu] 2 79349 Roberto Ricci <io@r-ricci.it> 79350 .IP \[bu] 2 79351 Bj\[/o]rn Smith <bjornsmith@gmail.com> 79352 .IP \[bu] 2 79353 Alishan Ladhani <8869764+aladh@users.noreply.github.com> 79354 .IP \[bu] 2 79355 zjx20 <zhoujianxiong2@gmail.com> 79356 .IP \[bu] 2 79357 Oksana <142890647+oks-maytech@users.noreply.github.com> 79358 .IP \[bu] 2 79359 Volodymyr Kit <v.kit@maytech.net> 79360 .IP \[bu] 2 79361 David Pedersen <limero@me.com> 79362 .IP \[bu] 2 79363 Drew Stinnett <drew@drewlink.com> 79364 .IP \[bu] 2 79365 Pat Patterson <pat@backblaze.com> 79366 .IP \[bu] 2 79367 Herby Gillot <herby.gillot@gmail.com> 79368 .IP \[bu] 2 79369 Nikita Shoshin <shoshin_nikita@fastmail.com> 79370 .IP \[bu] 2 79371 rinsuki <428rinsuki+git@gmail.com> 79372 .IP \[bu] 2 79373 Beyond Meat <51850644+beyondmeat@users.noreply.github.com> 79374 .IP \[bu] 2 79375 Saleh Dindar <salh@fb.com> 79376 .IP \[bu] 2 79377 Volodymyr <142890760+vkit-maytech@users.noreply.github.com> 79378 .IP \[bu] 2 79379 Gabriel Espinoza <31670639+gspinoza@users.noreply.github.com> 79380 .IP \[bu] 2 79381 Keigo Imai <keigo.imai@gmail.com> 79382 .IP \[bu] 2 79383 Ivan Yanitra <iyanitra@tesla-consulting.com> 79384 .IP \[bu] 2 79385 alfish2000 <alfish2000@gmail.com> 79386 .IP \[bu] 2 79387 wuxingzhong <qq330332812@gmail.com> 79388 .IP \[bu] 2 79389 Adithya Kumar <akumar42@protonmail.com> 79390 .IP \[bu] 2 79391 Tayo-pasedaRJ <138471223+Tayo-pasedaRJ@users.noreply.github.com> 79392 .IP \[bu] 2 79393 Peter Kreuser <logo@kreuser.name> 79394 .IP \[bu] 2 79395 Piyush 79396 .IP \[bu] 2 79397 fotile96 <fotile96@users.noreply.github.com> 79398 .IP \[bu] 2 79399 Luc Ritchie <luc.ritchie@gmail.com> 79400 .IP \[bu] 2 79401 cynful <cynful@users.noreply.github.com> 79402 .IP \[bu] 2 79403 wjielai <wjielai@tencent.com> 79404 .IP \[bu] 2 79405 Jack Deng <jackdeng@gmail.com> 79406 .IP \[bu] 2 79407 Mikubill <31246794+Mikubill@users.noreply.github.com> 79408 .IP \[bu] 2 79409 Artur Neumann <artur@jankaritech.com> 79410 .IP \[bu] 2 79411 Saw-jan <saw.jan.grg3e@gmail.com> 79412 .IP \[bu] 2 79413 Oksana Zhykina <o.zhykina@maytech.net> 79414 .IP \[bu] 2 79415 karan <karan.gupta92@gmail.com> 79416 .IP \[bu] 2 79417 viktor <viktor@yakovchuk.net> 79418 .IP \[bu] 2 79419 moongdal <moongdal@tutanota.com> 79420 .IP \[bu] 2 79421 Mina Gali\['c] <freebsd@igalic.co> 79422 .IP \[bu] 2 79423 Alen \[vS]iljak <dev@alensiljak.eu.org> 79424 .IP \[bu] 2 79425 \[u4F60]\[u77E5]\[u9053]\[u672A]\[u6765]\[u5417] <rkonfj@gmail.com> 79426 .IP \[bu] 2 79427 Abhinav Dhiman <8640877+ahnv@users.noreply.github.com> 79428 .IP \[bu] 2 79429 halms <7513146+halms@users.noreply.github.com> 79430 .IP \[bu] 2 79431 ben-ba <benjamin.brauner@gmx.de> 79432 .IP \[bu] 2 79433 Eli Orzitzer <e_orz@yahoo.com> 79434 .IP \[bu] 2 79435 Anthony Metzidis <anthony.metzidis@gmail.com> 79436 .IP \[bu] 2 79437 emyarod <afw5059@gmail.com> 79438 .IP \[bu] 2 79439 keongalvin <keongalvin@gmail.com> 79440 .IP \[bu] 2 79441 rarspace01 <rarspace01@users.noreply.github.com> 79442 .IP \[bu] 2 79443 Paul Stern <paulstern45@gmail.com> 79444 .IP \[bu] 2 79445 Nikhil Ahuja <nikhilahuja@live.com> 79446 .IP \[bu] 2 79447 Harshit Budhraja <52413945+harshit-budhraja@users.noreply.github.com> 79448 .IP \[bu] 2 79449 Tera <24725862+teraa@users.noreply.github.com> 79450 .IP \[bu] 2 79451 Kyle Reynolds <kylereynoldsdev@gmail.com> 79452 .IP \[bu] 2 79453 Michael Eischer <michael.eischer@gmx.de> 79454 .IP \[bu] 2 79455 Thomas M\[:u]ller <1005065+DeepDiver1975@users.noreply.github.com> 79456 .IP \[bu] 2 79457 DanielEgbers <27849724+DanielEgbers@users.noreply.github.com> 79458 .IP \[bu] 2 79459 Jack Provance <49460795+njprov@users.noreply.github.com> 79460 .IP \[bu] 2 79461 Gabriel Ramos <109390599+gabrielramos02@users.noreply.github.com> 79462 .IP \[bu] 2 79463 Dan McArdle <d@nmcardle.com> 79464 .IP \[bu] 2 79465 Joe Cai <joe.cai@bigcommerce.com> 79466 .IP \[bu] 2 79467 Anders Swanson <anders.swanson@oracle.com> 79468 .IP \[bu] 2 79469 huajin tong <137764712+thirdkeyword@users.noreply.github.com> 79470 .SH Contact the rclone project 79471 .SS Forum 79472 .PP 79473 Forum for questions and general discussion: 79474 .IP \[bu] 2 79475 https://forum.rclone.org 79476 .SS Business support 79477 .PP 79478 For business support or sponsorship enquiries please see: 79479 .IP \[bu] 2 79480 https://rclone.com/ 79481 .IP \[bu] 2 79482 sponsorship\[at]rclone.com 79483 .SS GitHub repository 79484 .PP 79485 The project\[aq]s repository is located at: 79486 .IP \[bu] 2 79487 https://github.com/artpar/artpar 79488 .PP 79489 There you can file bug reports or contribute with pull requests. 79490 .SS Twitter 79491 .PP 79492 You can also follow Nick on twitter for rclone announcements: 79493 .IP \[bu] 2 79494 [\[at]njcw](https://twitter.com/njcw) 79495 .SS Email 79496 .PP 79497 Or if all else fails or you want to ask something private or 79498 confidential 79499 .IP \[bu] 2 79500 info\[at]rclone.com 79501 .PP 79502 Please don\[aq]t email requests for help to this address - those are 79503 better directed to the forum unless you\[aq]d like to sign up for 79504 business support. 79505 .SH AUTHORS 79506 Nick Craig-Wood.