github.com/ncw/rclone@v1.48.1-0.20190724201158-a35aa1360e3e/docs/content/commands/rclone_dedupe.md (about) 1 --- 2 date: 2019-06-20T16:09:42+01:00 3 title: "rclone dedupe" 4 slug: rclone_dedupe 5 url: /commands/rclone_dedupe/ 6 --- 7 ## rclone dedupe 8 9 Interactively find duplicate files and delete/rename them. 10 11 ### Synopsis 12 13 14 By default `dedupe` interactively finds duplicate files and offers to 15 delete all but one or rename them to be different. Only useful with 16 Google Drive which can have duplicate file names. 17 18 In the first pass it will merge directories with the same name. It 19 will do this iteratively until all the identical directories have been 20 merged. 21 22 The `dedupe` command will delete all but one of any identical (same 23 md5sum) files it finds without confirmation. This means that for most 24 duplicated files the `dedupe` command will not be interactive. You 25 can use `--dry-run` to see what would happen without doing anything. 26 27 Here is an example run. 28 29 Before - with duplicates 30 31 $ rclone lsl drive:dupes 32 6048320 2016-03-05 16:23:16.798000000 one.txt 33 6048320 2016-03-05 16:23:11.775000000 one.txt 34 564374 2016-03-05 16:23:06.731000000 one.txt 35 6048320 2016-03-05 16:18:26.092000000 one.txt 36 6048320 2016-03-05 16:22:46.185000000 two.txt 37 1744073 2016-03-05 16:22:38.104000000 two.txt 38 564374 2016-03-05 16:22:52.118000000 two.txt 39 40 Now the `dedupe` session 41 42 $ rclone dedupe drive:dupes 43 2016/03/05 16:24:37 Google drive root 'dupes': Looking for duplicates using interactive mode. 44 one.txt: Found 4 duplicates - deleting identical copies 45 one.txt: Deleting 2/3 identical duplicates (md5sum "1eedaa9fe86fd4b8632e2ac549403b36") 46 one.txt: 2 duplicates remain 47 1: 6048320 bytes, 2016-03-05 16:23:16.798000000, md5sum 1eedaa9fe86fd4b8632e2ac549403b36 48 2: 564374 bytes, 2016-03-05 16:23:06.731000000, md5sum 7594e7dc9fc28f727c42ee3e0749de81 49 s) Skip and do nothing 50 k) Keep just one (choose which in next step) 51 r) Rename all to be different (by changing file.jpg to file-1.jpg) 52 s/k/r> k 53 Enter the number of the file to keep> 1 54 one.txt: Deleted 1 extra copies 55 two.txt: Found 3 duplicates - deleting identical copies 56 two.txt: 3 duplicates remain 57 1: 564374 bytes, 2016-03-05 16:22:52.118000000, md5sum 7594e7dc9fc28f727c42ee3e0749de81 58 2: 6048320 bytes, 2016-03-05 16:22:46.185000000, md5sum 1eedaa9fe86fd4b8632e2ac549403b36 59 3: 1744073 bytes, 2016-03-05 16:22:38.104000000, md5sum 851957f7fb6f0bc4ce76be966d336802 60 s) Skip and do nothing 61 k) Keep just one (choose which in next step) 62 r) Rename all to be different (by changing file.jpg to file-1.jpg) 63 s/k/r> r 64 two-1.txt: renamed from: two.txt 65 two-2.txt: renamed from: two.txt 66 two-3.txt: renamed from: two.txt 67 68 The result being 69 70 $ rclone lsl drive:dupes 71 6048320 2016-03-05 16:23:16.798000000 one.txt 72 564374 2016-03-05 16:22:52.118000000 two-1.txt 73 6048320 2016-03-05 16:22:46.185000000 two-2.txt 74 1744073 2016-03-05 16:22:38.104000000 two-3.txt 75 76 Dedupe can be run non interactively using the `--dedupe-mode` flag or by using an extra parameter with the same value 77 78 * `--dedupe-mode interactive` - interactive as above. 79 * `--dedupe-mode skip` - removes identical files then skips anything left. 80 * `--dedupe-mode first` - removes identical files then keeps the first one. 81 * `--dedupe-mode newest` - removes identical files then keeps the newest one. 82 * `--dedupe-mode oldest` - removes identical files then keeps the oldest one. 83 * `--dedupe-mode largest` - removes identical files then keeps the largest one. 84 * `--dedupe-mode rename` - removes identical files then renames the rest to be different. 85 86 For example to rename all the identically named photos in your Google Photos directory, do 87 88 rclone dedupe --dedupe-mode rename "drive:Google Photos" 89 90 Or 91 92 rclone dedupe rename "drive:Google Photos" 93 94 95 ``` 96 rclone dedupe [mode] remote:path [flags] 97 ``` 98 99 ### Options 100 101 ``` 102 --dedupe-mode string Dedupe mode interactive|skip|first|newest|oldest|rename. (default "interactive") 103 -h, --help help for dedupe 104 ``` 105 106 See the [global flags page](/flags/) for global options not listed here. 107 108 ### SEE ALSO 109 110 * [rclone](/commands/rclone/) - Show help for rclone commands, flags and backends. 111