github.com/artpar/rclone@v1.67.3/docs/content/commands/rclone_serve_ftp.md (about) 1 --- 2 title: "rclone serve ftp" 3 description: "Serve remote:path over FTP." 4 slug: rclone_serve_ftp 5 url: /commands/rclone_serve_ftp/ 6 groups: Filter 7 versionIntroduced: v1.44 8 # autogenerated - DO NOT EDIT, instead edit the source code in cmd/serve/ftp/ and as part of making a release run "make commanddocs" 9 --- 10 # rclone serve ftp 11 12 Serve remote:path over FTP. 13 14 ## Synopsis 15 16 17 Run a basic FTP server to serve a remote over FTP protocol. 18 This can be viewed with a FTP client or you can make a remote of 19 type FTP to read and write it. 20 21 ## Server options 22 23 Use --addr to specify which IP address and port the server should 24 listen on, e.g. --addr 1.2.3.4:8000 or --addr :8080 to listen to all 25 IPs. By default it only listens on localhost. You can use port 26 :0 to let the OS choose an available port. 27 28 If you set --addr to listen on a public or LAN accessible IP address 29 then using Authentication is advised - see the next section for info. 30 31 ### Authentication 32 33 By default this will serve files without needing a login. 34 35 You can set a single username and password with the --user and --pass flags. 36 37 ## VFS - Virtual File System 38 39 This command uses the VFS layer. This adapts the cloud storage objects 40 that rclone uses into something which looks much more like a disk 41 filing system. 42 43 Cloud storage objects have lots of properties which aren't like disk 44 files - you can't extend them or write to the middle of them, so the 45 VFS layer has to deal with that. Because there is no one right way of 46 doing this there are various options explained below. 47 48 The VFS layer also implements a directory cache - this caches info 49 about files and directories (but not the data) in memory. 50 51 ## VFS Directory Cache 52 53 Using the `--dir-cache-time` flag, you can control how long a 54 directory should be considered up to date and not refreshed from the 55 backend. Changes made through the VFS will appear immediately or 56 invalidate the cache. 57 58 --dir-cache-time duration Time to cache directory entries for (default 5m0s) 59 --poll-interval duration Time to wait between polling for changes. Must be smaller than dir-cache-time. Only on supported remotes. Set to 0 to disable (default 1m0s) 60 61 However, changes made directly on the cloud storage by the web 62 interface or a different copy of rclone will only be picked up once 63 the directory cache expires if the backend configured does not support 64 polling for changes. If the backend supports polling, changes will be 65 picked up within the polling interval. 66 67 You can send a `SIGHUP` signal to rclone for it to flush all 68 directory caches, regardless of how old they are. Assuming only one 69 rclone instance is running, you can reset the cache like this: 70 71 kill -SIGHUP $(pidof rclone) 72 73 If you configure rclone with a [remote control](/rc) then you can use 74 rclone rc to flush the whole directory cache: 75 76 rclone rc vfs/forget 77 78 Or individual files or directories: 79 80 rclone rc vfs/forget file=path/to/file dir=path/to/dir 81 82 ## VFS File Buffering 83 84 The `--buffer-size` flag determines the amount of memory, 85 that will be used to buffer data in advance. 86 87 Each open file will try to keep the specified amount of data in memory 88 at all times. The buffered data is bound to one open file and won't be 89 shared. 90 91 This flag is a upper limit for the used memory per open file. The 92 buffer will only use memory for data that is downloaded but not not 93 yet read. If the buffer is empty, only a small amount of memory will 94 be used. 95 96 The maximum memory used by rclone for buffering can be up to 97 `--buffer-size * open files`. 98 99 ## VFS File Caching 100 101 These flags control the VFS file caching options. File caching is 102 necessary to make the VFS layer appear compatible with a normal file 103 system. It can be disabled at the cost of some compatibility. 104 105 For example you'll need to enable VFS caching if you want to read and 106 write simultaneously to a file. See below for more details. 107 108 Note that the VFS cache is separate from the cache backend and you may 109 find that you need one or the other or both. 110 111 --cache-dir string Directory rclone will use for caching. 112 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off) 113 --vfs-cache-max-age duration Max time since last access of objects in the cache (default 1h0m0s) 114 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache (default off) 115 --vfs-cache-min-free-space SizeSuffix Target minimum free space on the disk containing the cache (default off) 116 --vfs-cache-poll-interval duration Interval to poll the cache for stale objects (default 1m0s) 117 --vfs-write-back duration Time to writeback files after last use when using cache (default 5s) 118 119 If run with `-vv` rclone will print the location of the file cache. The 120 files are stored in the user cache file area which is OS dependent but 121 can be controlled with `--cache-dir` or setting the appropriate 122 environment variable. 123 124 The cache has 4 different modes selected by `--vfs-cache-mode`. 125 The higher the cache mode the more compatible rclone becomes at the 126 cost of using disk space. 127 128 Note that files are written back to the remote only when they are 129 closed and if they haven't been accessed for `--vfs-write-back` 130 seconds. If rclone is quit or dies with files that haven't been 131 uploaded, these will be uploaded next time rclone is run with the same 132 flags. 133 134 If using `--vfs-cache-max-size` or `--vfs-cache-min-free-size` note 135 that the cache may exceed these quotas for two reasons. Firstly 136 because it is only checked every `--vfs-cache-poll-interval`. Secondly 137 because open files cannot be evicted from the cache. When 138 `--vfs-cache-max-size` or `--vfs-cache-min-free-size` is exceeded, 139 rclone will attempt to evict the least accessed files from the cache 140 first. rclone will start with files that haven't been accessed for the 141 longest. This cache flushing strategy is efficient and more relevant 142 files are likely to remain cached. 143 144 The `--vfs-cache-max-age` will evict files from the cache 145 after the set time since last access has passed. The default value of 146 1 hour will start evicting files from cache that haven't been accessed 147 for 1 hour. When a cached file is accessed the 1 hour timer is reset to 0 148 and will wait for 1 more hour before evicting. Specify the time with 149 standard notation, s, m, h, d, w . 150 151 You **should not** run two copies of rclone using the same VFS cache 152 with the same or overlapping remotes if using `--vfs-cache-mode > off`. 153 This can potentially cause data corruption if you do. You can work 154 around this by giving each rclone its own cache hierarchy with 155 `--cache-dir`. You don't need to worry about this if the remotes in 156 use don't overlap. 157 158 ### --vfs-cache-mode off 159 160 In this mode (the default) the cache will read directly from the remote and write 161 directly to the remote without caching anything on disk. 162 163 This will mean some operations are not possible 164 165 * Files can't be opened for both read AND write 166 * Files opened for write can't be seeked 167 * Existing files opened for write must have O_TRUNC set 168 * Files open for read with O_TRUNC will be opened write only 169 * Files open for write only will behave as if O_TRUNC was supplied 170 * Open modes O_APPEND, O_TRUNC are ignored 171 * If an upload fails it can't be retried 172 173 ### --vfs-cache-mode minimal 174 175 This is very similar to "off" except that files opened for read AND 176 write will be buffered to disk. This means that files opened for 177 write will be a lot more compatible, but uses the minimal disk space. 178 179 These operations are not possible 180 181 * Files opened for write only can't be seeked 182 * Existing files opened for write must have O_TRUNC set 183 * Files opened for write only will ignore O_APPEND, O_TRUNC 184 * If an upload fails it can't be retried 185 186 ### --vfs-cache-mode writes 187 188 In this mode files opened for read only are still read directly from 189 the remote, write only and read/write files are buffered to disk 190 first. 191 192 This mode should support all normal file system operations. 193 194 If an upload fails it will be retried at exponentially increasing 195 intervals up to 1 minute. 196 197 ### --vfs-cache-mode full 198 199 In this mode all reads and writes are buffered to and from disk. When 200 data is read from the remote this is buffered to disk as well. 201 202 In this mode the files in the cache will be sparse files and rclone 203 will keep track of which bits of the files it has downloaded. 204 205 So if an application only reads the starts of each file, then rclone 206 will only buffer the start of the file. These files will appear to be 207 their full size in the cache, but they will be sparse files with only 208 the data that has been downloaded present in them. 209 210 This mode should support all normal file system operations and is 211 otherwise identical to `--vfs-cache-mode` writes. 212 213 When reading a file rclone will read `--buffer-size` plus 214 `--vfs-read-ahead` bytes ahead. The `--buffer-size` is buffered in memory 215 whereas the `--vfs-read-ahead` is buffered on disk. 216 217 When using this mode it is recommended that `--buffer-size` is not set 218 too large and `--vfs-read-ahead` is set large if required. 219 220 **IMPORTANT** not all file systems support sparse files. In particular 221 FAT/exFAT do not. Rclone will perform very badly if the cache 222 directory is on a filesystem which doesn't support sparse files and it 223 will log an ERROR message if one is detected. 224 225 ### Fingerprinting 226 227 Various parts of the VFS use fingerprinting to see if a local file 228 copy has changed relative to a remote file. Fingerprints are made 229 from: 230 231 - size 232 - modification time 233 - hash 234 235 where available on an object. 236 237 On some backends some of these attributes are slow to read (they take 238 an extra API call per object, or extra work per object). 239 240 For example `hash` is slow with the `local` and `sftp` backends as 241 they have to read the entire file and hash it, and `modtime` is slow 242 with the `s3`, `swift`, `ftp` and `qinqstor` backends because they 243 need to do an extra API call to fetch it. 244 245 If you use the `--vfs-fast-fingerprint` flag then rclone will not 246 include the slow operations in the fingerprint. This makes the 247 fingerprinting less accurate but much faster and will improve the 248 opening time of cached files. 249 250 If you are running a vfs cache over `local`, `s3` or `swift` backends 251 then using this flag is recommended. 252 253 Note that if you change the value of this flag, the fingerprints of 254 the files in the cache may be invalidated and the files will need to 255 be downloaded again. 256 257 ## VFS Chunked Reading 258 259 When rclone reads files from a remote it reads them in chunks. This 260 means that rather than requesting the whole file rclone reads the 261 chunk specified. This can reduce the used download quota for some 262 remotes by requesting only chunks from the remote that are actually 263 read, at the cost of an increased number of requests. 264 265 These flags control the chunking: 266 267 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks (default 128M) 268 --vfs-read-chunk-size-limit SizeSuffix Max chunk doubling size (default off) 269 270 Rclone will start reading a chunk of size `--vfs-read-chunk-size`, 271 and then double the size for each read. When `--vfs-read-chunk-size-limit` is 272 specified, and greater than `--vfs-read-chunk-size`, the chunk size for each 273 open file will get doubled only until the specified value is reached. If the 274 value is "off", which is the default, the limit is disabled and the chunk size 275 will grow indefinitely. 276 277 With `--vfs-read-chunk-size 100M` and `--vfs-read-chunk-size-limit 0` 278 the following parts will be downloaded: 0-100M, 100M-200M, 200M-300M, 300M-400M and so on. 279 When `--vfs-read-chunk-size-limit 500M` is specified, the result would be 280 0-100M, 100M-300M, 300M-700M, 700M-1200M, 1200M-1700M and so on. 281 282 Setting `--vfs-read-chunk-size` to `0` or "off" disables chunked reading. 283 284 ## VFS Performance 285 286 These flags may be used to enable/disable features of the VFS for 287 performance or other reasons. See also the [chunked reading](#vfs-chunked-reading) 288 feature. 289 290 In particular S3 and Swift benefit hugely from the `--no-modtime` flag 291 (or use `--use-server-modtime` for a slightly different effect) as each 292 read of the modification time takes a transaction. 293 294 --no-checksum Don't compare checksums on up/download. 295 --no-modtime Don't read/write the modification time (can speed things up). 296 --no-seek Don't allow seeking in files. 297 --read-only Only allow read-only access. 298 299 Sometimes rclone is delivered reads or writes out of order. Rather 300 than seeking rclone will wait a short time for the in sequence read or 301 write to come in. These flags only come into effect when not using an 302 on disk cache file. 303 304 --vfs-read-wait duration Time to wait for in-sequence read before seeking (default 20ms) 305 --vfs-write-wait duration Time to wait for in-sequence write before giving error (default 1s) 306 307 When using VFS write caching (`--vfs-cache-mode` with value writes or full), 308 the global flag `--transfers` can be set to adjust the number of parallel uploads of 309 modified files from the cache (the related global flag `--checkers` has no effect on the VFS). 310 311 --transfers int Number of file transfers to run in parallel (default 4) 312 313 ## VFS Case Sensitivity 314 315 Linux file systems are case-sensitive: two files can differ only 316 by case, and the exact case must be used when opening a file. 317 318 File systems in modern Windows are case-insensitive but case-preserving: 319 although existing files can be opened using any case, the exact case used 320 to create the file is preserved and available for programs to query. 321 It is not allowed for two files in the same directory to differ only by case. 322 323 Usually file systems on macOS are case-insensitive. It is possible to make macOS 324 file systems case-sensitive but that is not the default. 325 326 The `--vfs-case-insensitive` VFS flag controls how rclone handles these 327 two cases. If its value is "false", rclone passes file names to the remote 328 as-is. If the flag is "true" (or appears without a value on the 329 command line), rclone may perform a "fixup" as explained below. 330 331 The user may specify a file name to open/delete/rename/etc with a case 332 different than what is stored on the remote. If an argument refers 333 to an existing file with exactly the same name, then the case of the existing 334 file on the disk will be used. However, if a file name with exactly the same 335 name is not found but a name differing only by case exists, rclone will 336 transparently fixup the name. This fixup happens only when an existing file 337 is requested. Case sensitivity of file names created anew by rclone is 338 controlled by the underlying remote. 339 340 Note that case sensitivity of the operating system running rclone (the target) 341 may differ from case sensitivity of a file system presented by rclone (the source). 342 The flag controls whether "fixup" is performed to satisfy the target. 343 344 If the flag is not provided on the command line, then its default value depends 345 on the operating system where rclone runs: "true" on Windows and macOS, "false" 346 otherwise. If the flag is provided without a value, then it is "true". 347 348 The `--no-unicode-normalization` flag controls whether a similar "fixup" is 349 performed for filenames that differ but are [canonically 350 equivalent](https://en.wikipedia.org/wiki/Unicode_equivalence) with respect to 351 unicode. Unicode normalization can be particularly helpful for users of macOS, 352 which prefers form NFD instead of the NFC used by most other platforms. It is 353 therefore highly recommended to keep the default of `false` on macOS, to avoid 354 encoding compatibility issues. 355 356 In the (probably unlikely) event that a directory has multiple duplicate 357 filenames after applying case and unicode normalization, the `--vfs-block-norm-dupes` 358 flag allows hiding these duplicates. This comes with a performance tradeoff, as 359 rclone will have to scan the entire directory for duplicates when listing a 360 directory. For this reason, it is recommended to leave this disabled if not 361 needed. However, macOS users may wish to consider using it, as otherwise, if a 362 remote directory contains both NFC and NFD versions of the same filename, an odd 363 situation will occur: both versions of the file will be visible in the mount, 364 and both will appear to be editable, however, editing either version will 365 actually result in only the NFD version getting edited under the hood. `--vfs-block- 366 norm-dupes` prevents this confusion by detecting this scenario, hiding the 367 duplicates, and logging an error, similar to how this is handled in `rclone 368 sync`. 369 370 ## VFS Disk Options 371 372 This flag allows you to manually set the statistics about the filing system. 373 It can be useful when those statistics cannot be read correctly automatically. 374 375 --vfs-disk-space-total-size Manually set the total disk space size (example: 256G, default: -1) 376 377 ## Alternate report of used bytes 378 379 Some backends, most notably S3, do not report the amount of bytes used. 380 If you need this information to be available when running `df` on the 381 filesystem, then pass the flag `--vfs-used-is-size` to rclone. 382 With this flag set, instead of relying on the backend to report this 383 information, rclone will scan the whole remote similar to `rclone size` 384 and compute the total used space itself. 385 386 _WARNING._ Contrary to `rclone size`, this flag ignores filters so that the 387 result is accurate. However, this is very inefficient and may cost lots of API 388 calls resulting in extra charges. Use it as a last resort and only with caching. 389 390 ## Auth Proxy 391 392 If you supply the parameter `--auth-proxy /path/to/program` then 393 rclone will use that program to generate backends on the fly which 394 then are used to authenticate incoming requests. This uses a simple 395 JSON based protocol with input on STDIN and output on STDOUT. 396 397 **PLEASE NOTE:** `--auth-proxy` and `--authorized-keys` cannot be used 398 together, if `--auth-proxy` is set the authorized keys option will be 399 ignored. 400 401 There is an example program 402 [bin/test_proxy.py](https://github.com/artpar/artpar/blob/master/bin/test_proxy.py) 403 in the rclone source code. 404 405 The program's job is to take a `user` and `pass` on the input and turn 406 those into the config for a backend on STDOUT in JSON format. This 407 config will have any default parameters for the backend added, but it 408 won't use configuration from environment variables or command line 409 options - it is the job of the proxy program to make a complete 410 config. 411 412 This config generated must have this extra parameter 413 - `_root` - root to use for the backend 414 415 And it may have this parameter 416 - `_obscure` - comma separated strings for parameters to obscure 417 418 If password authentication was used by the client, input to the proxy 419 process (on STDIN) would look similar to this: 420 421 ``` 422 { 423 "user": "me", 424 "pass": "mypassword" 425 } 426 ``` 427 428 If public-key authentication was used by the client, input to the 429 proxy process (on STDIN) would look similar to this: 430 431 ``` 432 { 433 "user": "me", 434 "public_key": "AAAAB3NzaC1yc2EAAAADAQABAAABAQDuwESFdAe14hVS6omeyX7edc...JQdf" 435 } 436 ``` 437 438 And as an example return this on STDOUT 439 440 ``` 441 { 442 "type": "sftp", 443 "_root": "", 444 "_obscure": "pass", 445 "user": "me", 446 "pass": "mypassword", 447 "host": "sftp.example.com" 448 } 449 ``` 450 451 This would mean that an SFTP backend would be created on the fly for 452 the `user` and `pass`/`public_key` returned in the output to the host given. Note 453 that since `_obscure` is set to `pass`, rclone will obscure the `pass` 454 parameter before creating the backend (which is required for sftp 455 backends). 456 457 The program can manipulate the supplied `user` in any way, for example 458 to make proxy to many different sftp backends, you could make the 459 `user` be `user@example.com` and then set the `host` to `example.com` 460 in the output and the user to `user`. For security you'd probably want 461 to restrict the `host` to a limited list. 462 463 Note that an internal cache is keyed on `user` so only use that for 464 configuration, don't use `pass` or `public_key`. This also means that if a user's 465 password or public-key is changed the cache will need to expire (which takes 5 mins) 466 before it takes effect. 467 468 This can be used to build general purpose proxies to any kind of 469 backend that rclone supports. 470 471 472 ``` 473 rclone serve ftp remote:path [flags] 474 ``` 475 476 ## Options 477 478 ``` 479 --addr string IPaddress:Port or :Port to bind server to (default "localhost:2121") 480 --auth-proxy string A program to use to create the backend from the auth 481 --cert string TLS PEM key (concatenation of certificate and CA certificate) 482 --dir-cache-time Duration Time to cache directory entries for (default 5m0s) 483 --dir-perms FileMode Directory permissions (default 0777) 484 --file-perms FileMode File permissions (default 0666) 485 --gid uint32 Override the gid field set by the filesystem (not supported on Windows) (default 1000) 486 -h, --help help for ftp 487 --key string TLS PEM Private key 488 --no-checksum Don't compare checksums on up/download 489 --no-modtime Don't read/write the modification time (can speed things up) 490 --no-seek Don't allow seeking in files 491 --pass string Password for authentication (empty value allow every password) 492 --passive-port string Passive port range to use (default "30000-32000") 493 --poll-interval Duration Time to wait between polling for changes, must be smaller than dir-cache-time and only on supported remotes (set 0 to disable) (default 1m0s) 494 --public-ip string Public IP address to advertise for passive connections 495 --read-only Only allow read-only access 496 --uid uint32 Override the uid field set by the filesystem (not supported on Windows) (default 1000) 497 --umask int Override the permission bits set by the filesystem (not supported on Windows) (default 2) 498 --user string User name for authentication (default "anonymous") 499 --vfs-block-norm-dupes If duplicate filenames exist in the same directory (after normalization), log an error and hide the duplicates (may have a performance cost) 500 --vfs-cache-max-age Duration Max time since last access of objects in the cache (default 1h0m0s) 501 --vfs-cache-max-size SizeSuffix Max total size of objects in the cache (default off) 502 --vfs-cache-min-free-space SizeSuffix Target minimum free space on the disk containing the cache (default off) 503 --vfs-cache-mode CacheMode Cache mode off|minimal|writes|full (default off) 504 --vfs-cache-poll-interval Duration Interval to poll the cache for stale objects (default 1m0s) 505 --vfs-case-insensitive If a file name not found, find a case insensitive match 506 --vfs-disk-space-total-size SizeSuffix Specify the total space of disk (default off) 507 --vfs-fast-fingerprint Use fast (less accurate) fingerprints for change detection 508 --vfs-read-ahead SizeSuffix Extra read ahead over --buffer-size when using cache-mode full 509 --vfs-read-chunk-size SizeSuffix Read the source objects in chunks (default 128Mi) 510 --vfs-read-chunk-size-limit SizeSuffix If greater than --vfs-read-chunk-size, double the chunk size after each chunk read, until the limit is reached ('off' is unlimited) (default off) 511 --vfs-read-wait Duration Time to wait for in-sequence read before seeking (default 20ms) 512 --vfs-refresh Refreshes the directory cache recursively in the background on start 513 --vfs-used-is-size rclone size Use the rclone size algorithm for Used size 514 --vfs-write-back Duration Time to writeback files after last use when using cache (default 5s) 515 --vfs-write-wait Duration Time to wait for in-sequence write before giving error (default 1s) 516 ``` 517 518 519 ## Filter Options 520 521 Flags for filtering directory listings. 522 523 ``` 524 --delete-excluded Delete files on dest excluded from sync 525 --exclude stringArray Exclude files matching pattern 526 --exclude-from stringArray Read file exclude patterns from file (use - to read from stdin) 527 --exclude-if-present stringArray Exclude directories if filename is present 528 --files-from stringArray Read list of source-file names from file (use - to read from stdin) 529 --files-from-raw stringArray Read list of source-file names from file without any processing of lines (use - to read from stdin) 530 -f, --filter stringArray Add a file filtering rule 531 --filter-from stringArray Read file filtering patterns from a file (use - to read from stdin) 532 --ignore-case Ignore case in filters (case insensitive) 533 --include stringArray Include files matching pattern 534 --include-from stringArray Read file include patterns from file (use - to read from stdin) 535 --max-age Duration Only transfer files younger than this in s or suffix ms|s|m|h|d|w|M|y (default off) 536 --max-depth int If set limits the recursion depth to this (default -1) 537 --max-size SizeSuffix Only transfer files smaller than this in KiB or suffix B|K|M|G|T|P (default off) 538 --metadata-exclude stringArray Exclude metadatas matching pattern 539 --metadata-exclude-from stringArray Read metadata exclude patterns from file (use - to read from stdin) 540 --metadata-filter stringArray Add a metadata filtering rule 541 --metadata-filter-from stringArray Read metadata filtering patterns from a file (use - to read from stdin) 542 --metadata-include stringArray Include metadatas matching pattern 543 --metadata-include-from stringArray Read metadata include patterns from file (use - to read from stdin) 544 --min-age Duration Only transfer files older than this in s or suffix ms|s|m|h|d|w|M|y (default off) 545 --min-size SizeSuffix Only transfer files bigger than this in KiB or suffix B|K|M|G|T|P (default off) 546 ``` 547 548 See the [global flags page](/flags/) for global options not listed here. 549 550 # SEE ALSO 551 552 * [rclone serve](/commands/rclone_serve/) - Serve a remote over a protocol. 553