github.com/xhghs/rclone@v1.51.1-0.20200430155106-e186a28cced8/docs/content/faq.md (about) 1 --- 2 title: "FAQ" 3 description: "Rclone Frequently Asked Questions" 4 date: "2015-08-27" 5 --- 6 7 Frequently Asked Questions 8 -------------------------- 9 10 ### Do all cloud storage systems support all rclone commands ### 11 12 Yes they do. All the rclone commands (eg `sync`, `copy` etc) will 13 work on all the remote storage systems. 14 15 ### Can I copy the config from one machine to another ### 16 17 Sure! Rclone stores all of its config in a single file. If you want 18 to find this file, run `rclone config file` which will tell you where 19 it is. 20 21 See the [remote setup docs](/remote_setup/) for more info. 22 23 ### How do I configure rclone on a remote / headless box with no browser? ### 24 25 This has now been documented in its own [remote setup page](/remote_setup/). 26 27 ### Can rclone sync directly from drive to s3 ### 28 29 Rclone can sync between two remote cloud storage systems just fine. 30 31 Note that it effectively downloads the file and uploads it again, so 32 the node running rclone would need to have lots of bandwidth. 33 34 The syncs would be incremental (on a file by file basis). 35 36 Eg 37 38 rclone sync drive:Folder s3:bucket 39 40 41 ### Using rclone from multiple locations at the same time ### 42 43 You can use rclone from multiple places at the same time if you choose 44 different subdirectory for the output, eg 45 46 ``` 47 Server A> rclone sync /tmp/whatever remote:ServerA 48 Server B> rclone sync /tmp/whatever remote:ServerB 49 ``` 50 51 If you sync to the same directory then you should use rclone copy 52 otherwise the two rclones may delete each others files, eg 53 54 ``` 55 Server A> rclone copy /tmp/whatever remote:Backup 56 Server B> rclone copy /tmp/whatever remote:Backup 57 ``` 58 59 The file names you upload from Server A and Server B should be 60 different in this case, otherwise some file systems (eg Drive) may 61 make duplicates. 62 63 ### Why doesn't rclone support partial transfers / binary diffs like rsync? ### 64 65 Rclone stores each file you transfer as a native object on the remote 66 cloud storage system. This means that you can see the files you 67 upload as expected using alternative access methods (eg using the 68 Google Drive web interface). There is a 1:1 mapping between files on 69 your hard disk and objects created in the cloud storage system. 70 71 Cloud storage systems (at least none I've come across yet) don't 72 support partially uploading an object. You can't take an existing 73 object, and change some bytes in the middle of it. 74 75 It would be possible to make a sync system which stored binary diffs 76 instead of whole objects like rclone does, but that would break the 77 1:1 mapping of files on your hard disk to objects in the remote cloud 78 storage system. 79 80 All the cloud storage systems support partial downloads of content, so 81 it would be possible to make partial downloads work. However to make 82 this work efficiently this would require storing a significant amount 83 of metadata, which breaks the desired 1:1 mapping of files to objects. 84 85 ### Can rclone do bi-directional sync? ### 86 87 No, not at present. rclone only does uni-directional sync from A -> 88 B. It may do in the future though since it has all the primitives - it 89 just requires writing the algorithm to do it. 90 91 ### Can I use rclone with an HTTP proxy? ### 92 93 Yes. rclone will follow the standard environment variables for 94 proxies, similar to cURL and other programs. 95 96 In general the variables are called `http_proxy` (for services reached 97 over `http`) and `https_proxy` (for services reached over `https`). Most 98 public services will be using `https`, but you may wish to set both. 99 100 The content of the variable is `protocol://server:port`. The protocol 101 value is the one used to talk to the proxy server, itself, and is commonly 102 either `http` or `socks5`. 103 104 Slightly annoyingly, there is no _standard_ for the name; some applications 105 may use `http_proxy` but another one `HTTP_PROXY`. The `Go` libraries 106 used by `rclone` will try both variations, but you may wish to set all 107 possibilities. So, on Linux, you may end up with code similar to 108 109 export http_proxy=http://proxyserver:12345 110 export https_proxy=$http_proxy 111 export HTTP_PROXY=$http_proxy 112 export HTTPS_PROXY=$http_proxy 113 114 The `NO_PROXY` allows you to disable the proxy for specific hosts. 115 Hosts must be comma separated, and can contain domains or parts. 116 For instance "foo.com" also matches "bar.foo.com". 117 118 e.g. 119 120 export no_proxy=localhost,127.0.0.0/8,my.host.name 121 export NO_PROXY=$no_proxy 122 123 Note that the ftp backend does not support `ftp_proxy` yet. 124 125 ### Rclone gives x509: failed to load system roots and no roots provided error ### 126 127 This means that `rclone` can't file the SSL root certificates. Likely 128 you are running `rclone` on a NAS with a cut-down Linux OS, or 129 possibly on Solaris. 130 131 Rclone (via the Go runtime) tries to load the root certificates from 132 these places on Linux. 133 134 "/etc/ssl/certs/ca-certificates.crt", // Debian/Ubuntu/Gentoo etc. 135 "/etc/pki/tls/certs/ca-bundle.crt", // Fedora/RHEL 136 "/etc/ssl/ca-bundle.pem", // OpenSUSE 137 "/etc/pki/tls/cacert.pem", // OpenELEC 138 139 So doing something like this should fix the problem. It also sets the 140 time which is important for SSL to work properly. 141 142 ``` 143 mkdir -p /etc/ssl/certs/ 144 curl -o /etc/ssl/certs/ca-certificates.crt https://raw.githubusercontent.com/bagder/ca-bundle/master/ca-bundle.crt 145 ntpclient -s -h pool.ntp.org 146 ``` 147 148 The two environment variables `SSL_CERT_FILE` and `SSL_CERT_DIR`, mentioned in the [x509 package](https://godoc.org/crypto/x509), 149 provide an additional way to provide the SSL root certificates. 150 151 Note that you may need to add the `--insecure` option to the `curl` command line if it doesn't work without. 152 153 ``` 154 curl --insecure -o /etc/ssl/certs/ca-certificates.crt https://raw.githubusercontent.com/bagder/ca-bundle/master/ca-bundle.crt 155 ``` 156 157 ### Rclone gives Failed to load config file: function not implemented error ### 158 159 Likely this means that you are running rclone on Linux version not 160 supported by the go runtime, ie earlier than version 2.6.23. 161 162 See the [system requirements section in the go install 163 docs](https://golang.org/doc/install) for full details. 164 165 ### All my uploaded docx/xlsx/pptx files appear as archive/zip ### 166 167 This is caused by uploading these files from a Windows computer which 168 hasn't got the Microsoft Office suite installed. The easiest way to 169 fix is to install the Word viewer and the Microsoft Office 170 Compatibility Pack for Word, Excel, and PowerPoint 2007 and later 171 versions' file formats 172 173 ### tcp lookup some.domain.com no such host ### 174 175 This happens when rclone cannot resolve a domain. Please check that 176 your DNS setup is generally working, e.g. 177 178 ``` 179 # both should print a long list of possible IP addresses 180 dig www.googleapis.com # resolve using your default DNS 181 dig www.googleapis.com @8.8.8.8 # resolve with Google's DNS server 182 ``` 183 184 If you are using `systemd-resolved` (default on Arch Linux), ensure it 185 is at version 233 or higher. Previous releases contain a bug which 186 causes not all domains to be resolved properly. 187 188 Additionally with the `GODEBUG=netdns=` environment variable the Go 189 resolver decision can be influenced. This also allows to resolve certain 190 issues with DNS resolution. See the [name resolution section in the go docs](https://golang.org/pkg/net/#hdr-Name_Resolution). 191 192 ### The total size reported in the stats for a sync is wrong and keeps changing 193 194 It is likely you have more than 10,000 files that need to be 195 synced. By default rclone only gets 10,000 files ahead in a sync so as 196 not to use up too much memory. You can change this default with the 197 [--max-backlog](/docs/#max-backlog-n) flag. 198 199 ### Rclone is using too much memory or appears to have a memory leak 200 201 Rclone is written in Go which uses a garbage collector. The default 202 settings for the garbage collector mean that it runs when the heap 203 size has doubled. 204 205 However it is possible to tune the garbage collector to use less 206 memory by [setting GOGC](https://dave.cheney.net/tag/gogc) to a lower 207 value, say `export GOGC=20`. This will make the garbage collector 208 work harder, reducing memory size at the expense of CPU usage. 209 210 The most common cause of rclone using lots of memory is a single 211 directory with thousands or millions of files in. Rclone has to load 212 this entirely into memory as rclone objects. Each rclone object takes 213 0.5k-1k of memory.