github.com/xhghs/rclone@v1.51.1-0.20200430155106-e186a28cced8/docs/content/faq.md (about)

     1  ---
     2  title: "FAQ"
     3  description: "Rclone Frequently Asked Questions"
     4  date: "2015-08-27"
     5  ---
     6  
     7  Frequently Asked Questions
     8  --------------------------
     9  
    10  ### Do all cloud storage systems support all rclone commands ###
    11  
    12  Yes they do.  All the rclone commands (eg `sync`, `copy` etc) will
    13  work on all the remote storage systems.
    14  
    15  ### Can I copy the config from one machine to another ###
    16  
    17  Sure!  Rclone stores all of its config in a single file.  If you want
    18  to find this file, run `rclone config file` which will tell you where
    19  it is.
    20  
    21  See the [remote setup docs](/remote_setup/) for more info.
    22  
    23  ### How do I configure rclone on a remote / headless box with no browser? ###
    24  
    25  This has now been documented in its own [remote setup page](/remote_setup/).
    26  
    27  ### Can rclone sync directly from drive to s3 ###
    28  
    29  Rclone can sync between two remote cloud storage systems just fine.
    30  
    31  Note that it effectively downloads the file and uploads it again, so
    32  the node running rclone would need to have lots of bandwidth.
    33  
    34  The syncs would be incremental (on a file by file basis).
    35  
    36  Eg
    37  
    38      rclone sync drive:Folder s3:bucket
    39  
    40  
    41  ### Using rclone from multiple locations at the same time ###
    42  
    43  You can use rclone from multiple places at the same time if you choose
    44  different subdirectory for the output, eg
    45  
    46  ```
    47  Server A> rclone sync /tmp/whatever remote:ServerA
    48  Server B> rclone sync /tmp/whatever remote:ServerB
    49  ```
    50  
    51  If you sync to the same directory then you should use rclone copy
    52  otherwise the two rclones may delete each others files, eg
    53  
    54  ```
    55  Server A> rclone copy /tmp/whatever remote:Backup
    56  Server B> rclone copy /tmp/whatever remote:Backup
    57  ```
    58  
    59  The file names you upload from Server A and Server B should be
    60  different in this case, otherwise some file systems (eg Drive) may
    61  make duplicates.
    62  
    63  ### Why doesn't rclone support partial transfers / binary diffs like rsync? ###
    64  
    65  Rclone stores each file you transfer as a native object on the remote
    66  cloud storage system.  This means that you can see the files you
    67  upload as expected using alternative access methods (eg using the
    68  Google Drive web interface).  There is a 1:1 mapping between files on
    69  your hard disk and objects created in the cloud storage system.
    70  
    71  Cloud storage systems (at least none I've come across yet) don't
    72  support partially uploading an object. You can't take an existing
    73  object, and change some bytes in the middle of it.
    74  
    75  It would be possible to make a sync system which stored binary diffs
    76  instead of whole objects like rclone does, but that would break the
    77  1:1 mapping of files on your hard disk to objects in the remote cloud
    78  storage system.
    79  
    80  All the cloud storage systems support partial downloads of content, so
    81  it would be possible to make partial downloads work.  However to make
    82  this work efficiently this would require storing a significant amount
    83  of metadata, which breaks the desired 1:1 mapping of files to objects.
    84  
    85  ### Can rclone do bi-directional sync? ###
    86  
    87  No, not at present.  rclone only does uni-directional sync from A ->
    88  B. It may do in the future though since it has all the primitives - it
    89  just requires writing the algorithm to do it.
    90  
    91  ### Can I use rclone with an HTTP proxy? ###
    92  
    93  Yes. rclone will follow the standard environment variables for
    94  proxies, similar to cURL and other programs.
    95  
    96  In general the variables are called `http_proxy` (for services reached
    97  over `http`) and `https_proxy` (for services reached over `https`).  Most
    98  public services will be using `https`, but you may wish to set both.
    99  
   100  The content of the variable is `protocol://server:port`.  The protocol
   101  value is the one used to talk to the proxy server, itself, and is commonly
   102  either `http` or `socks5`.
   103  
   104  Slightly annoyingly, there is no _standard_ for the name; some applications
   105  may use `http_proxy` but another one `HTTP_PROXY`.  The `Go` libraries
   106  used by `rclone` will try both variations, but you may wish to set all
   107  possibilities.  So, on Linux, you may end up with code similar to
   108  
   109      export http_proxy=http://proxyserver:12345
   110      export https_proxy=$http_proxy
   111      export HTTP_PROXY=$http_proxy
   112      export HTTPS_PROXY=$http_proxy
   113  
   114  The `NO_PROXY` allows you to disable the proxy for specific hosts.
   115  Hosts must be comma separated, and can contain domains or parts.
   116  For instance "foo.com" also matches "bar.foo.com".
   117  
   118  e.g.
   119  
   120      export no_proxy=localhost,127.0.0.0/8,my.host.name
   121      export NO_PROXY=$no_proxy
   122  
   123  Note that the ftp backend does not support `ftp_proxy` yet.
   124  
   125  ### Rclone gives x509: failed to load system roots and no roots provided error ###
   126  
   127  This means that `rclone` can't file the SSL root certificates.  Likely
   128  you are running `rclone` on a NAS with a cut-down Linux OS, or
   129  possibly on Solaris.
   130  
   131  Rclone (via the Go runtime) tries to load the root certificates from
   132  these places on Linux.
   133  
   134      "/etc/ssl/certs/ca-certificates.crt", // Debian/Ubuntu/Gentoo etc.
   135      "/etc/pki/tls/certs/ca-bundle.crt",   // Fedora/RHEL
   136      "/etc/ssl/ca-bundle.pem",             // OpenSUSE
   137      "/etc/pki/tls/cacert.pem",            // OpenELEC
   138  
   139  So doing something like this should fix the problem.  It also sets the
   140  time which is important for SSL to work properly.
   141  
   142  ```
   143  mkdir -p /etc/ssl/certs/
   144  curl -o /etc/ssl/certs/ca-certificates.crt https://raw.githubusercontent.com/bagder/ca-bundle/master/ca-bundle.crt
   145  ntpclient -s -h pool.ntp.org
   146  ```
   147  
   148  The two environment variables `SSL_CERT_FILE` and `SSL_CERT_DIR`, mentioned in the [x509 package](https://godoc.org/crypto/x509),
   149  provide an additional way to provide the SSL root certificates.
   150  
   151  Note that you may need to add the `--insecure` option to the `curl` command line if it doesn't work without.
   152  
   153  ```
   154  curl --insecure -o /etc/ssl/certs/ca-certificates.crt https://raw.githubusercontent.com/bagder/ca-bundle/master/ca-bundle.crt
   155  ```
   156  
   157  ### Rclone gives Failed to load config file: function not implemented error ###
   158  
   159  Likely this means that you are running rclone on Linux version not
   160  supported by the go runtime, ie earlier than version 2.6.23.
   161  
   162  See the [system requirements section in the go install
   163  docs](https://golang.org/doc/install) for full details.
   164  
   165  ### All my uploaded docx/xlsx/pptx files appear as archive/zip ###
   166  
   167  This is caused by uploading these files from a Windows computer which
   168  hasn't got the Microsoft Office suite installed.  The easiest way to
   169  fix is to install the Word viewer and the Microsoft Office
   170  Compatibility Pack for Word, Excel, and PowerPoint 2007 and later
   171  versions' file formats
   172  
   173  ### tcp lookup some.domain.com no such host ###
   174  
   175  This happens when rclone cannot resolve a domain. Please check that
   176  your DNS setup is generally working, e.g.
   177  
   178  ```
   179  # both should print a long list of possible IP addresses
   180  dig www.googleapis.com          # resolve using your default DNS
   181  dig www.googleapis.com @8.8.8.8 # resolve with Google's DNS server
   182  ```
   183  
   184  If you are using `systemd-resolved` (default on Arch Linux), ensure it
   185  is at version 233 or higher. Previous releases contain a bug which
   186  causes not all domains to be resolved properly.
   187  
   188  Additionally with the `GODEBUG=netdns=` environment variable the Go
   189  resolver decision can be influenced. This also allows to resolve certain
   190  issues with DNS resolution. See the [name resolution section in the go docs](https://golang.org/pkg/net/#hdr-Name_Resolution).
   191  
   192  ### The total size reported in the stats for a sync is wrong and keeps changing
   193  
   194  It is likely you have more than 10,000 files that need to be
   195  synced. By default rclone only gets 10,000 files ahead in a sync so as
   196  not to use up too much memory. You can change this default with the
   197  [--max-backlog](/docs/#max-backlog-n) flag.
   198  
   199  ### Rclone is using too much memory or appears to have a memory leak
   200  
   201  Rclone is written in Go which uses a garbage collector.  The default
   202  settings for the garbage collector mean that it runs when the heap
   203  size has doubled.
   204  
   205  However it is possible to tune the garbage collector to use less
   206  memory by [setting GOGC](https://dave.cheney.net/tag/gogc) to a lower
   207  value, say `export GOGC=20`.  This will make the garbage collector
   208  work harder, reducing memory size at the expense of CPU usage.
   209  
   210  The most common cause of rclone using lots of memory is a single
   211  directory with thousands or millions of files in.  Rclone has to load
   212  this entirely into memory as rclone objects.  Each rclone object takes
   213  0.5k-1k of memory.